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SUMMARY 



This final report describes the results of a project to conduct research 
on, and to develop, instructional materials for use in on-the-job train- 
ing of professional and non-professional library personnel in scientific 
and technical libraries. The work was conducted by System Development 
Corporation under contract with the U.S. Office of Education and with 
additional support by the U.S. Army ATLIS project. The project began 
on June 28, 1967, and was completed on May 15, 19^9. This report reviews 
the research, design, and development activities completed in previous 
reporting periods, but concentrates on the effort following October 31 » 
1968, which involved field testing the developed instruction, analyzing 
the test results, and making final modifications to the instructional 
packages before turning them over to USOE. 

Three instructional packages were developed. One, directed to profes- 
sional librarians, provided an introduction to system analysis, with 
particular emphasis on its relevance to library operations. This package 
consists of nine units of instruction, of which five combine the media of 
slides, sound, and workbook exercising, and four are presented as work- 
book materials. The other two instructional packages, designed for non- 
professional library personnel, consist of workbook materials on refer- 
ence tools and services and Russian-to -English transliteration. The 
Reference Tools and Services package contains 12 units of instruction. 

The Russian Transliteration package provides the equivalent of six such 
units, but is presented as a continuous sequence that requires a total 
of three to four hours of a trainee’s time, divided at his discretion* 

Each of the three instructional packages was tested in the field in 
libraries of different sizes and with junior- college students. The 
test design involved pre-testing to divide subjects into experimental 
and control groups and post-tests of the learning achieved by the two 
groups. In tests of the System Analysis and Reference Tools packages, 
the experimental groups were given the developed instruction, while the 
control groups were assigned to spend the same number of hours in self- 
directed study on the same subject matter. The Russian Transliteration 
differed in that two groups drilled on the same materials but one group’s 
sequence was randomized. The test results indicate significant differ- 
ences between experimental and control group learning for the System 
Analysis instruction and between the two treatment groups for the Russian- 
English Transliteration. There were smaller differences between the 
experimental and control groups for the Reference Tools and Services 
package instruction. We concluded from the results of the tests that 
the developed instruction meets its design objectives and provides 
effective means of enhancing skills in the three areas concerned. 
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1.0 INTRODUCTION 

This project is directed toward improving the knowledge and skills of 
library personnel through research in and development of on-the-job 
training courses. This has involved inquiry into past and current 
projects concerned with in-service training, educational and training 
theory and technique, identification of skills needed for library work, 
and planning for a broad program of instructional development. This 
project is intended r j illuminate the problems of building self-contained 
modular instructional packages designed to not require a personal 
instructor, and to explore the effectiveness of various media for impart- 
ing instruction to scientific and technical library personnel. An addi- 
tional. aim was that, while the initial course content would emphasize 
the special training needs of scientific and technical libraries, the 
design of the instruction would reflect the needs of other types of 
libraries so that the same instructional approaches, if successful, 
could be applied to their concerns. 

After the initial research and planning effort, the work of the project 
involved designing, constructing, pilot-testing, modifying, and evalu- 
ating experimental instructional materials, together with final modifica- 
tion in the light of testing, and turnover of the finished packages to the 
Office of Education. Three instructional packages were constructed. The 
first addresses the area of technology in libraries and consists of 9 
units of instruction in system analysis. Intended for professional 
library personnel, it emphasizes the perspectives and vocabulary of the 
system approach and seeks to facilitate communication between librarians 
and system personnel. The second package, consisting of 12 units of 
instruction in Reference Tools and Services, is intended for library 
assistants or technicians. This instruction emphasizes techniques of 
approaching unfamiliar reference works and attempts to impart, through 
precept and drill in specific works, a perspective and set of procedures 
applicable to reference work in general. The third instructional package 
concerns Russian-to-English transliteration. It presents the Russian 
alphabet, one letter at a time, with extensive drill on words containing 
only those letters that have already been presented — a simple version of 
programmed instruction. 

This final report is the last of four project reports. The three that 
have preceded it include TM-3762/000/01, "Planning for On-the-Job Train- 
ing of Library Personnel," which summarized the period July-December, 

1967 and outlined the activities of the project, and two interim reports, 
TM-3962/OOO/OO and TM-3962/OOl/OO, both titled "On-the-Job Training of 
Library Personnel." The two interim reports, covering the period Jan- 
uary to October 31, 1968, describe the design, construction, pilot- 
testing, modification of the instruction, and the experimental design 
for field testing the developed instruction. 
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Section 2 of this report summarizes the methods used in the research and 
development efforts previously reported; discusses the field test design, 
the selection of a sample subject population, and the conduct of the 
field tests; and summarizes some of the reactions of that population to 
the instruction. Sections 3 and 4 present the findings and analysis of 
the test data, and the conclusions and recommendations. The pre-tests 
and post-tests given to the subject population are presented in the 
Appendix. 



2.0 METHODS OF RESEARCH AMD DEVELOPMENT 

The general approach of this project has been to design instruction to 
assist in the mastery of skills, techniques, procedures, and other 
activities that are specific to a particular function or job in a scien- 
tific or technical library. We sought to ensure systematic coverage of 
materials and procedures, to test both alternative media and alternative 
techniques, and to test the efficacy of the finished instruction with a 
variety of persons. It was understood that to be effective, the means 
chosen for the instruction should be adaptable to individual differences 
in knowledge, learning capacity, and cognitive set., and should provide 
some sense of accomplishment and satisfaction in a reasonably short time. 
It was also understood that the field tests, because they were to be 
given immediately after the instruction was taken, could not test for 
the most important possible effects — those of long range improvement in 
skill, and the degree to which the immediate learning effects of the 
instruction are retained or attenuated over time. 

One important task of the project was to determine the objectives, or 
goals, of on-the-job training and of the course as a whole. Objectives 
were formulated primarily on the basis of the testimony of librarians 
interviewed early in the project; they were also influenced by a review 
of the relevant literature. The three objectives that were formulated 
regard (l) the training’s usefulness as a means of enhancing the skills 
and flexibility of library personnel, (2) its educational qualities, and 
(3) its cost, and were stated as follows: 

Personnel . The training should serve as a practical and 
effective means of introducing new information and techniques 
to library personnel regardless of their responsibility or 
professional skill level. It should also— by increasing the 
flexibility of personnel— alleviate the shortage of adequately 
trained personnel in the various library specialties and at 
various libraries. 
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Education . The training should encourage library personnel 
to view their jobs as— in part— learning experiences, and 
should establish the job setting as a learning environment. 

Further, it should create in those who undergo it a self- 
sufficiency with regard to how much they can (or should) 
learn, the approaches that aid their learning, and the degree 
to which they will seek out and participate in learning 
experiences; it should also stimulate in them a continuing 
desire to learn, and be useful to them in career planning, 
growth, and development. 

Cost . The training should achieve its personnel and educa- 
tional objectives without employing materials that are 
expensive to produce or that require expensive equipment, 
and without requiring that trainees spend unacceptable 
amounts of time away from their jobs. 

To ensure that these objectives would be met most effectively, 10 specific 
design and production requirements were stated! 

1. The format of all units in all packages should be consis- 
tent. This will help to ensure continuity in the manner 
and substance of the presentations; this continuity, in 
turn, will enable trainees to move easily and profitably 
from unit to unit and from package to package. 

2. The information that is to be mastered should be rein- 
forced in as many ways and at as many points as is 
necessary. 

3. The units within a package should be designed in the form 
of modules, so that a trainee can select the units he 
needs. 

h. Wherever it is both feasible and not unduly expensive, 
information should be presented in more than one medium. 

5. The training should require some active participation by 
the trainee. 

6. The training should be designed in such a way that the 
time, personnel, and equipment it requires can be real- 
istically allocated. 

7. The course packages should be designed for use during 
normal working hours. 



* 
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8. The use of the training packages should require a 
minimum of intervention by project staff members after 
they have installed them and briefed personnel on how 
to use them. 

9. The use of the packages should not usually require the 
participation, during the training session, by anyone 
other than the trainee. 

10. The packages should contain materials and references 
that will stimulate the trainee to augment his train- 
ing from other sources after he has mastered the 
material in the course packages themselves. 



2.1 REVIEW OF PREVIOUS REPORTING PERIODS 

2.1.1 Planning for the Project 

Between June and December, 1967, work on the project was devoted to 
reviewing educational and training literature, exploring in-service 
training needs in libraries of several kinds, and in writing a planning 
document for concurrence by the USOE and ATLIS. project monitors. During 
this period it was agreed that the planning document should comprehend 
a broad range of future training activities, but that the work of the 
current project should be limited to three areas— reference work, lan- 
guage, and technology in libraries. The planning document was submitted 
for review and concurrence on December 15, 1967, and was then distributed 
through ATLIS to a broad range of U.S, Government installations for 
comment and criticism. The project staff also sought reactions from non- 
Government libraries. The comments were largely interested, constructive 
and useful to the project. Concurrence on the plan was reached on 
February 7> 1968. 

2.1.2 Training Requirements and Design of Instruction 

Between February and the end of May, 1968, job and task requirements 
analyses were conducted for the areas of reference work, foreign and 
technical terminology, and technology in libraries. As the design effort 
proceedei, it was decided that the technology in libraries package would 
consist of an introduction to system analysis, emphasizing library appli- 
cations . It was also decided that the foreign and technical terminology 
instruction would consist of a package on Russian transliteration, and 
that the reference package would be devoted to reference tools and ser- 
vices. During this period, the project staff was materially aided by 
the advice and perspectives of the many librarians who were sought out 
for consultation on and reactions to these plans and designs. 
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The principal efforts during this period were devoted to developing 
formal statements of objectives for on-the-job training instruction and 
to detailing the design of instructional units within the three areas 
of concentration. Each unit was designed to be completed in 30-40 
minutes by a typical trainee, with the expectation that some trainees 
would complete the units more quickly and others might take as long as 
50 or 60 minutes. The work also included developing specific design 
and production requirements, including, in some detail, an outline of 
the content to be covered in each unit and the media to be used for 
presenting it. 

2.1.3 Package Construction. Pilot Testing, and Field Test Design 

From May 31 to October 31, 1968, the design and development of the three 
packages continued, and pilot tryouts— involving an iterative process of 
preliminary testing, subsequent modification, and retesting of instruc- 
tional material — began. In addition, effort was devoted to completing 
an experimental design for field testing the completed instruction. 

As work on the packaged proceeded, it became clear that the instruction 
could be improved if the original projection of the number and kinds of 
units to be constructed for the System Analysis and Reference packages 
were revised. It was originally thought, for example , that two multi- 
media units would be sufficient for the System Analysis package, with 
the addition of a sound and workbook unit on the problems of communica- 
tion between librarians and systems personnel. In the end, nine units 
were constructed— five multi-media units consisting of slides, sound, 
and workbook exercises, and four workbook-only units. (Figure 1 lists 
the major topics covered in the nine units.) This, of course, increased 
the work involved considerably. A story line built about a realistic 
library situation had to be carefully scripted, many more photographs 
than had been originally planned had to be taken, and rehearsals were 
necessary for the speakers of the sound portion of the instruction. 

The number of units devoted to reference tools and services was also 
revised. In order to concentrate more effectively on that area we 
decided not to produce any units directed to interaction with users but 
confine the construction to reference books. The final list of units 
in the Reference Tools and Services package is shown in Figure 2. 

The field test designs were described earlier, in detail, in TM-39&2 / 
001/00 ; they will be summarized here. Each package required a somewhat 
different test procedure. The System Analysis package, which was tested 
on experienced professional librarians who had no prior experience with 
system analysis, required an experimental and a control group. The 
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TECHNOLOGY IN LIBRARIES 
AN INTRODUCTION TO SYSTEM ANALYSIS 



Unit Is The System Approach and Basic Terminology 

Unit 2: Communicating with Systems Personnel 

Objectives, Requirements and Constraints 

Unit 3: Phase Analysis, Functional Analysis and Task Analysis 

Unit 4: Allocation of Resources and Cost-Effectiveness 

Unit 5: System Analysis and Decision Making 

Unit 6: Applying the System Approach to Problems 

Unit 7s Phase and Functional Analysis 

Unit 8: Task Analysis 

Unit 9: Utilizing Analysis Data 



Figure 1. Major Topics in the System Analysis Instruction 



REFERENCE TOOLS AND SERVICES 



INTRODUCTION TO REFERENCE WORKS 
BIBLIOGRAPHIC TOOLS - SERIALS 

Ulrich's International Periodicals Directory 
Union List of Serials in the United States and Canada 
New Serial Titles 
COMMERCIAL DIRECTORIES 

Thomas Register of American Manufacturers 
GENERAL TECHNICAL ENCYCLOPEDIAS 

McGraw-Hill Encyclopedia of Science and Technology 
ENGINEERING HANDBOOKS 

Standard Handbook for Mechanical Engineers 
Handbook of the Engineering Sciences 
ORGANIZATION DIRECTORIES 

Research Centers Directory 
BIOGRAPHICAL DIRECTORIES 

American Men of Science 
SERIALS INDEXING AND ABSTRACTING SERVICES 
Engineering Index 
Physics Abstracts 

REPORTS INDEXING AND ABSTRACTING SERVICES 

U.S. Government Research and Development Reports 



Figure 2. Reference Works Studied by Experimental and Control Groups 
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testing sequence for both groups consisted of a pre-test of their knowl- 
edge of system analysis, a training sequence, a post-test of their 
knowledge of system analysis, and a test of their ability to interpret 
+?° nn+Pht Of an article written from the system standpoint. The 
Reference Tools and Services test design included the administration of 
severS kinds of tests, used to divide the control and experimental sub- 
SuSation into matched groups. It employed a content-knowledge 
ire!t!stf^ instruction sequence, and a post-test. The testing of the 
Russian Transliteration package did not require a control group, as 
such- rather, it consisted of two groups of subjects matched by individ- 
ual-difference pre-testing. The groups were given different kinds of 
i^t motion. One was given the programmed instruction; the other was 
given 1 the same drill materials but in a randomized sequence, undatable 
designed to assist memorization. All tests for the transliteration sub- 
lectf were timed. The primary statistical tools for analyzing the data 
for all three experiments were analyses of variance and covariance. 

a final task during this period was to make arrangements for the field 
testing with cooperating libraries and students from Santa Monica City 
Sf . The libraries that participated in the field testing included 
the National Agricultural Library, the Ios Angeles Public Library, 
Edge^TsenSlechnical library, Picatinny Arsenal Technical Infor- 
mation Branch, Bunker-Kamo Corporation Library, the University of 
California at San Diego ' s Engineering library , the Marine Technical 
I^mSion CenSr oAhe California Bepartment of Fish and Game, and 
the Base library of the Presidio of San Francisco. 



2.2 



COMPLETING THE PROJECT— NOVEMBER 1, 1968-MAY 15 > 1969 



2.2.1 Field Testing the Instruction 

The first steps in preparing for the field tests were to determine the 
characteristics of the subject population and divide them into expen- 
mental^andcontrol groups, and to schedule the field tests in such a 
_____ that the timing of the pre-testing, instruction, and post-testing 
was similar for all subjects despite their widely scattered locations. 

Subjects in the cooperating libraries were selected by the library 
staffs . 

For the svstem analysis instruction, we asked the libraries to provide 
different kinds ofprof essionals , as available, who had no experience 
with system analysis. We also asked that each subject have a profes- 
lional degreftn librarianship. We felt that intelligence and interest 
testing would be unnecessary for this population, since there wo 
already have been systematic selection by education and job category, 
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but we did gather some biographical information from these subjects. The 
subjects for the experimental group were drawn from the National Agricul- 
tural Library and from the libraries at Edgewood Arsenal, Picatinny 
Arsenal, and the Presidio at San Francisco; all but the Presidio Base 
Library are technical libraries. The control group was drawn from the 
Los Angeles Public Library. The experimental group received the six- 
hour System Analysis package; the control group was directed to spend the 
same number of hours in self-directed study of system analysis and was 
supplied with a suggested list of books and serial titles for guidance. 

All of the experimental subjects were asked to comment on their personal 
reactions to the instruction before they were given the post-tests. This 
part of the experiment occupied the period January-March, 19o9« 

Subjects for testing the Reference Tools and Services and Transliteration 
packages consisted on non-professional library personnel and students. 

The control group included non-professionals from the National Agricul- 
tural Library. The experimental group included non-professionals from 
Edgewood Arsenal, Picatinny Arsenal, the National Agricultural Library, 
Bunker-Ramo Corporation Library, the Engineering Library of the Univer- 
sity of California at Los Angeles, and the Marine Technical Information 
Center of the California Department of Fish and Game. Since the two 
packages were designed to train persons who had little or no formal 
library training, we felt that it was appropriate to use in both groups 
additional subjects who had a minimum of previous experience with the 
subject matter of the instruction. Therefore, 30 female Santa Monica 
City College students were used, since their preparation is comparable 
to that of new trainees employed in libraries at the library assistant 
or technician level; moreover, their behavior under the experimental con- 
ditions could be more readily controlled than that oi the library per- 
sonnel. 

In addition to the pre-tests addressed to the content of the instruction, 
the subjects were given three tests to divide them into control and 
experimental groups. These were the Wonderlic Personnel Test Form 1, 
the Strong Vocational Interest Blank for Women ^ librarian Scale;, and a 
truncated version of the Library Orientation Test for College Freshmen, 
published by Teacher’s College Press. The Strong Vocational Interest 
Blank proved not to be useful for this purpose, and was not generally 
presented to the library personnel subjects. The tests were begun in 
January and completed in May, 1969. The control group was directed to 
spend nine hours— the same amount of time required for the instruction— 
in self-directed study of the same reference works. As with the System 
Analysis instruction, personal reactions were sought from the experimental 

group. 
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The Russian Transliteration field tests were begun with the students from 
Santa Monica City College in November, 1968, and finished with subjects 
provided by the National Agricultural Library. As has been indicated 
earlier, two matched groups were given different kinds of instruction. 

One was given the programmed sequence, in which one Russian letter is 
presented at a time with extensive drill materials confined to letters 
that have already been presented. The second group was given a table of 
the Russian-English equivalents designed to assist memorization, and a 
randomized sequence of the same drill materials given the first group. 

This latter situation was intended to simulate that of anyone’s having 
to learn the Russian alphabet using a table of Russian-English equivalents 
and such practice materials as come to hand. Using the identical drill 
materials ensured a more severe test of the effectiveness of the pro- 
grammed sequence than could be expected from using a completely self- 
directed control group. 

Two kinds of difficulties were encountered in the field testing. The 
first was that there was some attrition in subjects that had begun to 
take the sequences of instruction. Certain of the subjects who had been 
assigned to groups on the basis of several kinds of pre-tests dropped 
away before finishing, and substitutes had to be sought and found to take 
their places where possible. We do not believe that the substitutions 
are likely to have had significant effects, however, because the numbers 
of substitutions involved was small (4). A more serious difficulty was 
encountered in retaining the integrity of the control groups. The control 
groups were much smaller than the experimental groups for the reference 
tools and system analysis instruction experiment. We had hoped to have 
a minimum of 15 in each control group. Unfortunately, we were left with 
only 12 in the control group for the reference tools instruction, and 11 
for the system analysis instruction. 

Another difficulty, which does not relate directly to the experiment but 
which may have considerable implications for the feasibility of on-the- 
job training of the kind that this project has built, was that most of 
the libraries participating in the experiment had some difficulty in pro- 
viding the time required for their personnel to take the instruction. In 
several cases , the time devoted to the instruction was spread over many 
more weeks than was typical of the general experimental population. These 
problems reflect the pressure of work and the short staffing in those 
several libraries. This is in no sense a criticism of the libraries 
participating in this experiment. Rather, it is a confirmation of the 
shortage of library personnel and the prevalence of understaffing. It is 
probable that most libraries do not have sufficient manpower to allow 
adequate on-the-job training for systematic career development. It is 
clear, however, that if appropriate training is to be afforded in librar- 
ies and their staffs brought to higher levels of skill, training time 
must be budgeted for and allocated as a part of normal operations . 
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2.2.2 Final Modification and Packaging of the Instruction 

Several changes were made in all three instructional packages as a result 
of the field testing, comments from those who took the instruction, and 
our own error-detection procedures. The Russian Transliteration package 
had been divided into appropriate units so that each would occupy a sub- 
ject for 30 to 40 minutes. As revised, the package now forms one sequence 
of simple programmed instruction, and leaves it to the trainee to decide 
how much time to spend and how much to attempt in any one study session. 
Also, several improvements were made in the instruction and some of the 
exercise materials were revised and augmented. 

Revision to the Reference Tools and Services package were confined pri- 
mal ily to rectifying typographical errors and mistaken page references 
in two of the units. There were comments to the effect that more instruc- 
tion in the area of reports literature would be very valuable. There was 
some indication on the part of a minority of library subjects that there 
were more exercises in one or two of the units than was needed for 
mastery of the material. We felt, however, that trainees in the field 
would not be required to go further in exercising on practice material 
than they thought necessary. 

Several modifications were necessary to the system analysis instruction, 
particularly in the workbook materials, where some of the definitions and 
explanations were not completely clear. It was necessary, also, to 
reshoot many of the slides because of inadequate quality of reproduction 
in processing. 

In addition, the multi-media packages for the system analysis instruction 
had required the use of a stereo tape recorder and a Carousel 800 slide 
projector, together with an adapter for transmitting signals from the 
tape to the projector. This was an easily procured, flexible arrangement 
suitable for experimental purposes, providing projection on a wall or 
screen for numbers of persons at once, or, with the use of the Sawyer- 
Mirascreen, for one or two people at a time at a desk. Also, there were 
provisions for using either loudspeakers or earphones. This arrangement, 
however, is not optimum for unaided field use. Therefore we sought to 
find a simpler method of presenting material through filmstrip. 

After surveying equipment currently on the narket, we decided to make two 
filmstrips as a part of the five units of instruction to be turned over 
to USOE, that would be designed to be used with the DuKane Cassette A-V- 
Matic sound filmstrip projector. This equipment appears to be most 
satisfactory for individual use in that it is simple to use mechanically 
and provides a desk-mounted mechanism with a 9 x 7” projector screen 
built-in. The filmstrip could, of course, be displayed on other 
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projector equipment. Filmstrips offer many advantages over slides in 
that they cannot he put out of sequence and are available for projection 
on a variety of machines. The Cassette, as a carrier of magnetically 
recorded sound, has become an industry standard and should also be widely 
adaptable to different makes of projection equipment. Finally, if large 
numbers of copies of the instruction are desired, the filmstrip offers 
many economies over slides for volume production. 



3.0 FINDINGS AND ANALYSIS 

The results of the training experiments on the three instructional 
packages will be analyzed and discussed separately. We will first treat 
the system analysis training experiment, second the reference tool train- 
ing experiment, and finally the transliteration training experiment. 
Selection of subjects for the experiments has been discussed earlier in 
Section 2.2.1. 

3.1 SYSTEM ANALYSIS TRAINING STUDY 

The system analysis training experiment was performed with an experimental 
group of 22 subjects and a control group of 11 subjects. The test pro- 
cedures were as follows : 

1. Both groups received an identical pre-test of content 
knowledge in the area of system analysis. This pre-test 
contained 43 multiple- choice items. 

2. The experimental group received the nine system analysis 
training units (six hours of instruction). 

3. The control group spent six hours in self -directed study 
of supplied reference materials. 

4. After the training period, both groups were given a post- 
test of their knowledge of system analysis. This test 
contained 72 items, of which 43 were identical with the 
43-item pre-test. 

5. Both groups were asked to read a short article drawn 
from the system analysis literature and to take a 25-item 
test of their comprehension of the article's content. 

This test was intended to measure the extent to which 
the subjects’ reading of the article was facilitated by 
the instruction and the self-study. 
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The summary statistical data for the experimental and control groups on 
the pre-test and on the 43 repeated items of the post-test are presented 
in Table 1. 



Table 1. System Analysis Pre-Test and Post-Test Scores 



Pre-test 



Experimental N = 22 Control N = 11 





M 


20.3 


19.5 




s 


4.2 


3.3 


Post -test 


(Repeat Items) 


M 


26.2 


21.0 




s 


4.9 


3.9 


Difference. 


Correlated 


D 


5-9 


1.5 


Means (p) 


est a 


• 95 


.83 



The information in the table includes the mean and standard deviation of 
the distribution of scores for each group for each test, the mean differ- 
ence scores between pre-tests and post-tests, and the standard errors of 
the mean differences. The first statistical test made was between the 
pre-test and repeated-items scores of both groups. The standard error 
of a correlated difference between means was computed directly from dif- 
ference scores. For the experimental group, the one-tailed t-test of the 
hypothesis that learning occurred yielded a t-value of 6.24 (p«.0005). 

For 21 degrees of freedom, this value of t is well beyond accepted 
"extremely high" significance levels. The estimate of the percentage of 
the variance attributable to training was 47.5% for the experimental group. 
The identical comparison for the control group yielded a t-value of I.85, 
with 10 degrees of freedom, which is below conventional significance 
levels. Thus, while learning probably occurred in both groups, it was 
not statistically significant in the control group, but highly significant 
in the experimental group. 

To confirm these results, a test of the "difference between the differences" 
was performed. A linear combination of the difference scores was formed 
to reflect the differences in learning between the two groups, and the 
standard error of the linear combination was computed. (This linear 
combination is analogous to the interaction term in analysis of variance. ) 
The value of the linear combination was U.05 with a standard deviation 
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of 1.26, yielding a t-value of 3.21 Tor the test of the hypothesis of 
the linear combination equaling 0 (a hypothesis equivalent to the "No 
interaction" hypothesis in analysis of variance). The hypothesis was 
rejected with a high degree of confidence: (p < .005). The interpretation 

of these three results is that: 

(a) The experimental group exhibited a high degree of learning. 

(b) The control group exhibited a low degree of learning. 

(c) The difference in results between the two groups was very 
significant. 

A further comparison, summarized in Table 2, was made to confirm these 
results using the total post-test scores . (This comparison is not 
entirely independent of the preceding comparisons because of the items 
in common.) The obtained value of t for a one-tailed test of the 
hypothesis of superiority of the experimental group yielded a t-value of 
4.03 (p < .005), which supports the thesis that the experimental group 
performed considerably better on this test than did the control group. 

The estimate of the percentage of variance attributable to training was 
32°/ 0 , which is somewhat lower than the estimate made from the difference 
scores. (However, while the difference scores were effectively corrected 
for individual differences on the pre-test, these differences could not 
be corrected in the analysis of the total post-test scores, and therefore 
contributed to the error variance of the latter test.) 



Table 2. System Analysis Total Post-Test Scores 





Experimental 


Control 


M 


48.1 


39.2 


5 


6.7 


5.5 


eSt a M M 


2.21 





The statistical results of the reading- facilitation test, summarized in 
Table 3, were inconclusive. The control group had a slightly higher 
score, but the difference between groups yielded a t-value of 1.48, 
which is below conventional levels of significance. That is, while the 
earlier content test showed the expected learning differences, the 
reading- facilitation test did not. One explanation of these results is 
that the major effect of an introductory training sequence such as the 
System Analysis package is to increase one’s knowledge of the vocabulary 
of the content area. 
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Table 3. Systems Analysis Reading Test Scores 





Experimental 


Control 


M 


15.6 


17.1 


s 


3.9 


1.9 


A 






est a 


1.038 




M-M 










In conclusion, results of the System Analysis training experiment are 
positive. For both the test-retest and the post-test conparisons of 
the slides, sound, and workbook experimental package and the self- 
directed- reading control treatment, the findings were highly favorable 
to the slide-sound-and-paper package. However, the results from the 
reading-facilitation test probably constitute a factual reminder of the 
limits of what can reasonably be expected to be acconplished by a short 
course of study in an abstract, unfamiliar, wide-ranging subject matter; 
the six hours of slide, sound, and workbook instruction, while superior 
to self-directed reading, did not produce deep understanding of system 
analysis. Comments and testimony gathered from the subjects, however, 
suggested that the training package had succeeded in transforming system 
analysis into a topic perceived as approachable, manageable, and of new 
interest to the subjects. 

3.2 REFERENCE TOOLS TRAINING STUDY 

The reference tools experiment was performed with an experimental group 
of 28 and a control group of 12 subjects. The procedure was similar to 
that used for the system analysis training experiments, in that both 
groups were first given identical pre-tests, and, after their treatment, 
identical post-tests that repeated all the items on the pre-tests plus 
additional items. However, there was no reading- facilitation test for 
this experiment. The pre-test contained 38 multiple choice items. The 
post-test contained 60 items that included three fill-in questions. 

Table 4 summarizes the scores for both groups on the pre-test, for both 
groups on those items on the post-test that were repeated pre-test items , 
and the difference scores. Standard deviations are given for the scores, 
and the standard error of the mean difference is given with the difference 
scores. 
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Table 4. Reference Tools Pre-Test and Post-Test Scores 







Experimental N = 28 


Control N = 12 


Pre-test 


M 


18.7 


17.9 




s 


5.3 


5.4 


Post-test 






21.5 

1.2 


(Repeat Items) 


M 

s 


23.0 

5.5 


Difference, 




* 


3.58 


Correlated 


D 


4.25 


Means (D) 








est 


A 

0 _ 


.81 


1.25 



D 



The first comparison was for difference between pre-test scores and 
repeated- items scores. The experimental group yielded a t-value of 
5.25 (p «.0005), which is beyond the ’’extremely significant” level. 

The estimate of the amount of variance attributable to training was 
32j. A similar test for the difference between correlated means for 
the control group yielded a t-value of 2.88 (p <.0l), a "very significant” 
difference. The estimate of the percentage of variance attributable to 
training was 2 4$. Finally, a test of the "difference between differences” 
was performed, using the same linear combination scheme that was used for 
the system analysis. Since the hypothesis that the linear combination 
equaled zero yielded a t-value less than one , it was not rejected. 

These results show that considerable learning occurred. The experimental 
group's superiority over the control group is not resoundingly clear, but 
it is evident from the data that hot a groups did learn a considerable 
amount, as indicated by the percentages of the variance attributable to 
training. A final test of the differences between groups, summarized 
in Table 5, was performed using the total post-test scores. In this case, 
a test of the difference between non-correlated means yielded a t-value 
of 1.45 (.10 < p < .05), which indicates a slight superiority for the 
experimental group. 
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Table 5. Reference Tools Total Post-Test Scores 



Experimental 



Control 



M 



s 

A 



31.5 

8.5 



27.8 

5.9 



est a 



2.42 



M-M 



Concluding, the most significant finding of the reference tools training 
experiment relates to the test-retest results. The experimental group's 
performance was at least marginally better than the control group, even 
with the limited sample sizes that we were able to enploy in this experi- 
ment. Since the same source materials and equal study times were 
available to both experimental and control groups, it is reasonable to 
conclude that the added directions and exercises of the experimental 
treatment did contribute something to the learning. 

3.3 TRANSLITERATION TRAINING STUDY 

The Transliteration Package experiment was performed on two groups. The 
first group, which received sequenced training material, numbered 15. 

The second group, which received the same training material but without 
the sequencing, numbered l4. These groups were equated on a 4-minute 
pre-test of transliteration ability and several other tests that are 
referred to in Section 2.2.1. A post-test was given to both groups. 

The test was a 15-minute timed test containing 184 Russian words. Two 
scores were recorded--the total number of attempted items and an error 



score. 



These scores are summarized in Tables o and 7> respectively. 



Table 6. Transliteration Items Attempted 



Sequenced Group N = 15 



Non- Sequenced Group N = 14 



M 



s 



144.9 

32.9 




est a 



M-M 



12.7 
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Considering the total number of items attempted in Table 6, there is 
no appreciable difference in either the mean scores or the standard 
deviations. The t-value for the difference between means was less than 
1. Thus, in number of items attempted there was virtually no difference 
between the two training methods. 



Table 7* Transliteration Error Scores 





Sequenced Group Hon- Sequenced Group 


M 


15.13 31.85 


s 


9*5 32.8 


A 

est a 


9^5 


M-M 





The error score in Table 7, however, indicates a marked difference 
between the two groups. A test of difference between mean error scores 
yielded a t-value of 1.77 (p < .05) for a one-tailed test of the 
hypothesis that the sequenced training material was superior to the 
random material. In addition, it can be noted that the group using the 
random material were extremely variable in their error scores, with a 
standard deviation of 32.8 as compared to a standard deviation of 9*5 
for the sequenced- instruction group. This increased variability in- 
flated the standard error of difference between the means; therefore, 
the. t-value is somewhat smaller than might be expected if the groups 
were homogeneous with respect to their variability. This difference 
in variability between the two groups is, however, further evidence of 
the superiority of the sequenced training; different individuals in the 
group that studied the randomly sequenced material apparently had very 
different learning effects, a few subjects learning the material fairly 
well, but others apparently learning little. In sum, the error rate 
for the random-material group was twice that of the sequenced-material 
group, and this was partly because more subjects in the first group did 
very poorly. 

The conclusions for the Transliteration Package training experiment are 
positive. Both experimental treatments undoubtedly represent considerably 
more effective training regimes than the sporadic experiences with isolated 
bits and pieces of transliteration that are typical of informal trans- 
literation training in most library environments. The more structured 
of the two training treatments, in which careful consideration was given 
to the sequence of instruction, resulted in appreciably lower error rates 
in the test performances. This finding should be viewed in the light of 
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the typically limited training time available for producing transliteration 
skills, the sporadic nature of the typical demand for transliteration; and 
the importance of accuracy in transliteration. Given these considerations, 
the carefully sequenced, package of instruction is a valuable resource. 

3.4 SUBJECTS’ PERSONAL REACTIONS TO THE INSTRUCTION 

Immediately before we gave the post-tests on the three packages of 
instruction, we sought personal reactions to the instruction from those 
who had completed it. We were interested in eliciting indications of 
interest or disinterest in the material and media as presented, as well 
as identifying the instruction's strengths and weaknesses as seen by the 
trainees. We wanted both to relate the subjects’ impressions to the 
test results and to use these impressions to improve the instruction, 
where appropriate. 

In inviting subjective impressions and reactions, as well as criticism, 
we obtained a broad range of responses, as might be expected; these 
responses did not, in fact, correlate well with the test results. For 
exanple, the reactions of professional librarians who took the system 
analysis instruction ranged from enthusiasm to distaste, with the 
preponderance of comment being critical both in general and in 
detail. Many of the "destructive" comments were just and accurate. 

Still, the tests of content mastery indicate that significant learning 
was achieved. In contrast, comments on the reference tools instruction 
was largely complimentary, approving, and constructive; the test results, 
however, show a less appreciable gap between significant learning in the 
experimental and control groups. The two groups exposed to different 
treatments in the Russian transliteration experiment expressed interest 
in the tasks to which they were put and testified almost unanimously that 
they found the instruction interesting. Here, again, there was a 
significant difference in the learning performance tested between the 
two groups, though both groups learned to transliterate well. 

lf.O CONCLUSIONS AND RECOMMENDATIONS 

The major objective of this project has been to design, construct, and 
test modular packages of instruction that would improve the skills of 
library personnel without requiring a personal instructor. This objective 
has been met. This is not to say that the finished instruction is perfect, 
that the presence of a personal instructor would not be helpful, nor that 
the instruction is effective for all personnel. The work of the project 
has demonstrated the feasibility of building self-contained packages that 
do improve skills in a practical and useful manner. 
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A secondary objective of the project was to explore different media and 
techniques for presenting instruction. The System Analysis package 
exploited the media of slides, sound, and workbook exercising to present 
abstract ideas in several ways, and used self -testing under time con- 
straints as an instructional tool. This permitted the presentation of 
many complex concepts and perspectives in a rather short sequence of 
instructions that would exploit several sense modalities to sustain 
interest in the trainees and be responsive to the differential ability 
of individuals to learn from different media. As has been noted, the 
subjective reactions of the experimental subjects to the instruction in 
the field tests varied from enthusiasm to distaste, while the test re- 
sults show quite significant learning for most of the subjects. We 
conclude that the multimedia approach to teaching system analysis is 
effective and could, with the experience gained here, be usef ully 
extended to a more advanced treatment of the subject matter. 

The instruction directed to reference tools experimented with severs! 
approaches to presenting doctrine, procedure, and drill, and, like the 
system analysis instruction, used diagnostic self -testing to alert the 
trainee to what he was expected to learn. The test results show 
significant learning in both the experimental and control groups with 
not very significant differences between the two groups. 

The testimony given by the experimental subjects indicated a much more 
preponderant interest in the subject content and liking for the 
instruction, than was characteristic of the system analysis sequence. 

We conclude that the use of several approaches and techniques does 
enhance interest and would be appropriate for any extension of in- 
structional development in chis area. 

The Russian Transliteration package explored a simple programmed learning 
approach as compared with a simulated situation of self -directed study. 

The results show that the experimental subjects learned effectively and 
that they found the instruction interesting. We conclude that a 
programmed approach is an effective and efficient manner of teaching 
English transliteration of strange alphabets. 

In Siam, we find that the work of this project has pursued a viable line 
of inquiry and development that should be extended to additional areas 
of need for on-the-job training of library personnel. A logical extension 
of the work begun by the project would include: 

1. Extension of the system analysis instruction at a more advanced 
level that would focus on functional and task analyses and 
their relationships to design decisions and managerial decisions, 
with emphasis on the quantitative aspects of data analysis, 
budgeting, and costing; 
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2. Development of a sequence on reference tools and services, 
intended for library- assistant or technician-grade personnel 
in non-technical libraries , that would emphasize materials 
held in small public libraries and the services such libraries 
perform; 

3. Development of a sequence on microforms and reprography that 
would instruct library personnel how to use the materials, 
processes, and equipment in this area, and how to choose among 
them for particular applications; and 

4. A study of the longer- range impact of instruction than was 
possible under the current project to provide means of assessing 
the effects— both motivational and informational— of packaged 
instruction, and the degree to which further learning and skill 
is developed by such training. 
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APPENDIX 



The following pages contain the pre- and post- tests 
for the three instructional packages. These include 
the pre-test and two post-tests for the System Analysis 
instruction, the Reference Tools and Services* pre-test 
and post- test, and the comparable tests for the Russian- 
English Transliteration instruction. 
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PRE-TEST 



for 



SYSTEM ANALYSIS 



January 1969 







All but one of the questions on the following pages are multiple choice. 
For each question circle the letter in front of the word or phrase pro- 
viding the "best" answer. 
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1. In system analysis the main emphasis is: 

a. mathematical in orientation 

b. concerned with digital computers 

c. methodical application of common sense 

d. all the above 

e. none of the above 

2. The statement "To provide high quality reference service to 
community X" is an example of a system: 

a. objective 

b . limit 

c. constraint 

d. resource 

e. all the above 

3. The statement "The yearly book budget will not exceed $21,000" 
is an example of a system: 

a. objective 

b. limit 

c. constraint 

d. resource 

e. all the above 

4. The process of "iteration" in system analysis refers to: 

a. a mathematical solution to a problem 

b. a declarative statement made at a given point in analysis 

c. repeated cycling of analysis at greater level of detail 

d. all the above 

e. none of the above 

5. In the system development sequence: 

a. system analysis follows design, with a sharp separation 

b. design follows system analysis, with a sharp separation 

c. system analysis tends to follow design, with considerable 
overlap 

d. design tends to follow system analysis, with considerable 
overlap 
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In the overall field of system analysis: 

a. there is standardized terminology, which is important 

b. there is NO completely general standardized terminology 
for the whole field 

c. standardized terminology is essential within any one 
project 

d. both b and c are true 

The techniques of system analysis: 

a. are applicable mainly to modern automated systems 

b. are in fact restricted to computer-based systems 

c. are equally applicable to non-automated manual systems 

d. almost always involve the use of linear programming 

The organization of system components and functions into an 
operating whole is based on: 

a. regular interaction between various parts and various 
functions 

b. overlapping of objectives for various parts and functions 

c. sharing of resources by various parts and functions 

d. all of the above 

Parts of systems that perform major functions are called: 

a. subsystems 

b . components 

c. interfaces 

d. none of the above 

In system analysis parlance, descriptions of lines of authority, 
physical facilities, and locations are: 

a. functional descriptions 

b. structural descriptions 

c. relational descriptions 

d. none of the above 

The most direct connection in the systan description is between 
objectives and: 



a. functions 

b. phases 

c. tasks 

d. interfaces 
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12. Each item listed below is either a system objective (0), system 

requirement (R) , system limit (L) , or system constraint (C) . 

Put the correct letter designation in each of the blanks. 

"Personnel manning cannot exceed 11 persons." 

"Yearly budget cannot exceed $87,000.00." 

"Budgeting cycle starts in July." 

"High grade information service will be supplied to 

the research divisions." 

"Complete administrative information will be generated 

for the library managers." 

"Complete records will be kept for all items flowing 

through the system." 

"The system must be capable of processing 20,000 

acquisitions per year." 

"$16,000.00 per year will be needed for purchase of 

hardcovers." 

"Reference service for 1300 persons in the engineering 

division must be supplied." 

"Selective alerting function must be available for 60 

top managers." 

"80 hours of IBM 360/67 time are needed per year." 

"Average delay for copies of journal articles delivered 

to office should not exceed one working day." 

"System must be capable of processing a minimum of 30 

typical user requests per day." 

13. Functional analysis asks the question: 



a. 


How? 




b . 


To what 


end? 


c. 


When? 




d. 


all the 


above 


e. 


none of 


the above 



14. A detailed study of the activities supporting a system function 
is called: 

a. a functional analysis 

b. a task analysis 

c. a phase analysis 

d. a trade-off analysis 

e. none of the above 
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15. Managing a system and managing change and development of the 
system: 

a. , should be one and the same thing 

b. should be done by the same person 

c. are distinct tasks that should be kept separate 

d. both a and b are correct 

e. none of the above are true 



16. A system requirement may be defined as: 

a. a generalized statement of system purpose 

b. an authorized and specifically described need 

c. a necessary mechanical relation between system parts 

d. none of the above 

17. In system development, the most common sequence of events 
would be: 



18. 



19. 



20 . 



a. analysis, design, test, evaluate 

b. design, analysis, text, evaluate 

c. test, analysis, evaluate, design 

d. evaluate, design, test, analysis 

As compared to system objectives, system requirements are: 

a. more specifically stated 

b. less specifically stated 

c. formally authorized 

d. both a and c are true 

e. none of the above are true 

System criteria are: 

a. directly relevant to system objectives 

b. expressed in quantitative terms 

c. expressed in observable terms 

d. all of the above are true 

e. none of the above are true 

In system analysis, system documentation is concerned with: 

a. citing supporting literature for decisions 

b. detailed descriptions of system objective and constraints 

c. detailed descriptions of system limits, functions, phases 

tasks 

d. recording the inquiries, reasoning, and findings of the 
analysis 

e. all the above 
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21. The flow of information across an interface goes: 

a. from input to input 

b. from output to input 

c. from input to output 

d. none of the above 

22. In system analysis, the level of detail of description tends 
to be: 

a. greater for functions than for objectives 

b. greater for objectives than for functions 

c. greater for tasks than for objectives 

d. both a and c are true 

23. A system analysis often shows that: 

a. one function serves several objectives 

b. one objective is served by several functions 

c. there is only one function for each objective 

d. both a and b are true 

e. none of the above are true 

24. System requirements are: 

a. another name for system limits 

b. another name for system constraints 

c. another name for system objectives 

d. all the above are true 

e. none of the above are true 

25. Some of the benefits of "mass production" can often be 
achieved by : 

a. function consolidation 

b. operational definitions of tasks 

c. system decentralization 

d. installing computers 

26. It would be most accurate to say that: 

a. System effectiveness must be spelled out before system 
benefits can be detailed. 

b. System benefits must be spelled out before system 
effectiveness can be measured. 

c. System costs must be known before system benefits can 
be assessed. 

d. Value trade-off functions must be known before system 
benefits can be identified. 
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27. To estimate system effectiveness requires: 

a. data on system costs 

b. data on system benefits 

c. value trade-off functions for benefits 

d. both b and c are true 

e. all the above are true 

28. A system benefit is quantified or measured by: 

a. its cost 

b. counting or rating of defining events 

c. the value trade-off function for the benefit 

d. none of the above 

29. The most useful common denominator measure for system costs 
is probably: 



a. 


personnel 


time 


b. 


personnel 


effort 


c. 


personnel 


inconvenience 


d. 


money 





30. The purpose of (decision tables) in system analysis is: 

a. to specify what decision is indicated by the data 

b. to focus the decision on effectiveness factors 

c. to focus the decision on cost factors 

d. to portray cost-effectiveness balances 

31. In almost all system analyses: 

a. Most system costs can be added and subtracted directly. 

b. Amounts of various system benefits can be added and 
subtracted directly. 

c. Value trade-off functions for benefits need not even be 
considered. 

d. Cost-effectiveness analyses comprise the preliminary 
step . 
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All but one of the questions on the following pages are multiple choice. 
For each question circle the letter in front of the word or phrase pro- 
viding the "best" answer. 
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1. In system analysis the main emphasis is: 

a. mathematical in orientation 

b. concerned with digital computers 

c. methodical application of common sense 

d. all the above 

e. none of the above 

2. A system analysis of a library would be unlikely to use knowledge 
from the field of: 

a. decision theory 

b. time and motion studies 

c. attitude measurement 

d. statistical mechanics 

e. all the above 

3. The statement "To provide high quality reference service to 
community X" is an example of a system: 

a. objective 

b . limit 

c. constraint 

d. resource 

e. all the above 

4. The statement "The reference service must be able to handle a 
minimum of twenty requests per day" is an example of a system: 

a. objective 

b. limit 

c. constraint 

d. resource 

e. all the above 

5. The statement "The yearly book budget will not exceed $21,000" is 
an example of a system: 

a. objective 

b. limit 

c. constraint 

d. resource 

e. all the above 
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6. The statement "Interlibrary loan arrangements are in effect with 
the main library at city Y" is an example of a system: 

a. objective 

b. limit 

c. constraint 

d. resource 

e. all the above 

7. The process of "iteration" in system analysis refers to: 

a. a mathematical solution to a problem 

b. a declarative statement made at a given point in analysis 

c. repeated cycling of analysis at greater level of detail 

d. all of the above 

e. none of the above 

8. One of the greatest strengths of system analysis as a technique 
for understanding a system is: 

a. its selection of only certain details for study 

b. its elimination of the need for great detail 

c. its relating of each detail to the overall picture of the 
system 

d. its practice of considering each detail in isolation from 
all others 

9. In the system development sequence: 

a. system analysis follows design, with a sharp separation 

b. design follows system analysis, with a sharp separation 

c. system analysis tends to follow design, with considerable 
overlap . 

d. design tends to follow system analysis, with considerable 
overlap 

The relationships between parts of a system are organized through 
consideration of: 

a. system objectives 

b. system limits 

c. system resources 

d. all the above 



10 . 



15 May 1969 



40 



TM- 3962/002/00 



11. In the overall field of system analysis: 

a. there is standardized terminology, which is important 

b. there is no completely general standardized terminology, 
for the whole field 

c. standardized terminology is essential within any one project 

d. both b and c are true 

12. In analyzing a system, the sequence of analysis tends to be: 

a. objectives, functions, structure 

b. structure, objectives, functions 

c. functions, objectives, structure 

d. structure, functions, objectives 

13. The techniques of system analysis: 

a. are applicable mainly to modern automated systems 

b. are in fact restricted to computer-based systems 

c. are equally applicable to non-automated manual systems 

d. almost always involve the use of linear programming 

14. The organization of system components and functions into an 
operating whole is based on: 

a. regular interaction between various parts and various functions 

b. overlapping of objectives for various parts and functions 

c. sharing of resources by various parts and functions 

d. all of the above 

15. The terms "system", "subsystem", and "component" are: 

a. functional descriptions 

b. structural descriptions 

c. hierarchically related (some subsume others) 

d. both b and c are true 

16. Parts of systems that perform major functions are called: 

a. subsystems 

b. components 

c. interfaces 

d. none of the above 
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A system component is: 

a. a task performed by one or more parts 

b. a function performed by one or more parts 

c! a structural part that performs one or more tasks 
dl a structural part that performs one or more functions 

In system analysis parlance, descriptions of lines of authority, 
physical facilities, and locations are: 

a. functional descriptions 

b. structural descriptions 

c. relational descriptions 

d. none of the above 

It would be most accurate to say that in doing system analysis. 

a. functions must be understood before structure can be perceived 

b. structure must be perceived before functions can be understood 

c. structural arrangements should depend on function require- 
ments 

d. functional requirements should flow from structural arrange- 
ments 

The most direct connection in the system description is between 
objectives and: 

a. functions 

b. phases 

c. tasks 

d. interfaces 

It would be most accurate to say that in doing system analysis: 

a. system objectives are deduced from system structure 

b. system functions are deduced from system objectives 

c. system tasks follow directly from system objectives 

d. system components are assigned directly to system objectives 
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22. Each item listed below is either a system objective, (0) system 

requirement, (R) system limit, (L) or system constraint (C) . Put 

the correct letter designation in each of the blanks. 

"Personnel manning cannot exceed 11 persons." 

"Yearly budget cannot exceed $87,000.00." 

"Budgeting cycle starts in July." 

"High grade information service will be supplied to the 

research divisions." 

"Complete administrative information will be generated for the 

library managers." 

"Complete records will be kept for all items flowing through 

the system." 

"The system must be capable of processing 20,000 acquisitions 

per year." 

"$16,000.00 per year will be needed for purchase of hard-covers." 

"Reference service for 1,300 persons in the engineering division 

must be supplied." 

"Selective alerting function must be available for 60 top managers. 

"80 hours of IBM 360/67 time are needed per year." 

"Average delay for copies of journal articles delivered to office 

should not exceed one working day." 

"System must be capable of processing a minimum of 30 typical user 

requests per day." 

23. In system terminology, a target date for a deliverable end item in 

design or development is called a: 

a. due date 

b. milestone 

c. data item 

d. none of these 

24. Functional analysis asks the question: 



a. 


How? 




h* 


To what 


end? 


c. 


When? 




d. 


all the 


above 


e. 


none of 


the above 



25. Phase analysis concentrates on: 

a. the purposes of system activity 

b. sequence of information flow 

c. activities supporting the information flow 

d. stage of system development 

e. none of the above 
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26. A detailed study of the activities supporting a system function is 
cal led : 

a. a functional analysis 

b. a task analysis 

c. a phase analysis 

d. a tradeoff analysis 

e. none of the above 



27. A system analysis usually is concerned with: 

• a. existent system conditions 

b. projected system conditions 

c. environmental trends 

d. all of the above 

e. none of the above 

28. Managing a system and managing change and development of the system: 

* 

a. should be one and the same thing 

b. should be done by the same person 

c. are distinct tasks that should be kept separate 

d. both a and b are correct 

e. none of the above are true 

29. The system analysis technique: 

a. depends only on observations from the existing system 

b. employs extrapolation and simulation as tools 

c. uses information and specifications from the literature 

d. both b and c are correct 

e. none of the above are true 

30. A system requirement may be defined as: 

a. a generalized statement of system purpose 

b. an authorized and specifically described need 

c. a necessary mechanical relation between system parts 

d. none of the above 



In system development. 



the most common sequence of events would be: 



a. 

b. 

c. 

d. 



analysis, design, test, evaluate 
design, analysis, test, evaluate 
test, analysis, evaluate, design 
evaluate, design, test, analysis 



o 

ERIC 



31. 
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32. In system analysis, documentation of the process and its product 
is considered: 

a. a desirable practice that must be considered in terms of 
funds available 

b. mainly 

c. a waste of time and funds 

d. absolutely essential for preserving the benefits of the 
analysis 

e. purely a matter of individual tastes 

f. not vital since what counts is the analyst's grasp of the 
system 

33. As compared to system objectives, system requirements are: 

a. more specifically stated 

b. less specifically stated 

c. formally authorized 

d. both a and c are true 

e. none of the above are true 

34. As compared to system objectives, system criteria are: 

a. expressed more in concrete terms 

b. expressed more in terms of observable events 

c. expressed more in terms of quantifiable observations 

d. all of the above are true 

e. none of the above are true 



35. System criteria are: 

a. directly relevant to system objectives 

b. expressed in quantitative terms 

c. expressed in observable terms 

d. all the above are true 

e. none of the above are true 

36. Operational definitions are: 

a. specific descriptions of actions required to observe or 
measure a defined event 

b. very important to system analysis 

c. highly abstract (in distinction to concrete) definitions 

d. both a and b are true 
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37. In system analysis, system documentation is concerned with: 

a. citing supporting literature for decisions 

b. detailed descriptions of system objectives and constraints 

c. detailed descriptions of system limits, functions, phases 
and tasks 

d. recording the inquiries, reasoning, and findings of the 
analys is 

e. all the above 

38. The concept of inputs and outputs applies to: 



a. 


any system 


b. 


subsys terns 


c. 


components 


d. 


all the above 


e. 


none of the above 



39. The flow of information across an interface goes: 

a. from input to input 

b. from output to input 

c. from input to output 

d. none of the above 

40. System boundaries are defined: 

a. only by inputs 

b. only by outputs 

c. by both inputs and outputs 

d. only by task analysis 

41. Almost all system analyses consist partly of: 

a. functional analysis, and phase analysis 

b. task analysis and tradeoff analysis 

c. computer programming and statistical analysis 

d. both a and b are true 

In system analysis, the level of detail of description tends to be 

a. greater for functions than for objectives 

b. greater for objectives than for functions 

c. greater for tasks than for objectives 

d. both a and c are true 



42 . 
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43. A system analysis often shows that: 

a. one function serves several objectives 

b. one objective is served by several functions 
0 ' there is only one function for each objective 

d. both a and b are true 

e. none of the above are true 

44. It would be most accurate to say that: 

a. functions rationalize (i.e., provide the basis for) objectives 

b. objectives rationalize functions 

c. tasks rationalize phases 

d. tasks rationalize functions 

45. System requirements are: 

a. another name for system limits 

b. another name for system constraints 

c. another name for system objectives 

d. all the above are true 

e. none of the above are true 

46. System analysis documentation would ordinarily not include 
descriptions of: 

a. system objectives and requirements 

b. acceptance testing results for the system 

c. system constraints and limits 

d. system phases, functions, and tasks 

47. Some of the benefits of "mass production" can often be achieved by: 



a. function consolidation 

b. operational definitions of tasks 

c. system decentralization 

d. installing computers 

48. Cost-effectiveness analysis can be helpful for: 

a. making resource-allocation decisions 

b. checking task analyses . ... 

c. checking implementation of resource-allocation decisions 

d. both a and c are true 
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49. System analysts tend to view a good decision as one which: 

a. no one can argue about 

b. accurately anticipates the outcome of future events 

c. is the most popular with powerful decision makers 

d. requires the most sophisticated technical system design 

50. It would be most accurate to say that: 

a. system effectiveness must be spelled out before system 
benefits can be detailed 

b. system benefits must be spelled Cut before system effective- 
ness can be measured 

c. system costs must be known before system benefits can be 
assessed 

d. value tradeoff functions must be known before system benefits 
can be identified 

51. Information about system costs is obtained: 

a. from cost figures for the system 

b. from literature about similar systems 

c. by extrapolation and estimation procedures 

d. all the above are true 

e. none of the above are true 



52. To estimate system effectiveness requires: 



a. data on system costs 

b. .V ta on system benefits 

c. /alue tradeoff functions for benefits 

d. both b and c are true 

e. all the above are true 



53. Proper resource allocation decisions are based on: 

a. system cost figures 

b. system benefits measurements 

c. value tradeoff functions for benefits 

d. both b and c are true 

e. all the above 



54. A system benefit is quantified or measured by: 



a. its cost 

b. counting or rating of defining events 

c. the value tradeoff function for the benefit 

d. none of the above 
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55. Information about the comparative values of various system benefits 
can best be obtained: 



a. 

b. 

c. 

d. 



from system cost analyses 
from the general system’s literature 
from system personnel and users 
from the system analyst 



56. The most useful common denominator measure for system costs is 
probably: 



a. 

b. 



personnel time 
personnel effort 
personnel inconvenience 
money 



57. It would be most accurate to say that: 



a. measuring system costs is always harder than measuring 
system benefits 

b. the amount and value of a benefit are essentially the 
same thing 

c. costs and benefits are sometimes two ways of looking at 
the same thing 

d. there is no way to measure the value of a benefit 



58. The purpose of (decision tables) in system analysis is: 



a. 

b. 

c. 

d. 



to specify what decision is indicated by the data 
to focus the decision on effectiveness factors 
to focus the decision on cost factors 
to portray cost effectiveness balances 



59. When various system alternatives are closely competitive: 



a. cost and effectiveness measures must be made more explicit 

b. value tradeoff functions must be more explicit 

c. more raw intuition is usually the most useful 

d. both a and b are true 
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60. In almost all system analyses: 

a. most system costs can be added and subtracted directly 

b. amounts of various system benefits can be added and 
subtracted directly 

c. value tradeoff functions for benefits need not even be 
considered 

a. cost-effectiveness analyses comprise the preliminary step 
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POST-TEST 



for 



SYSTEM ANALYSIS 



Part 2 



January 1969 



Testing Instructions : Circle the single best answer for each of the 25 items. 
The questions should be answered from the point of view of the authors of the 
article. You will have 30 minutes to answer the 25 questions. 

This test is based upon the article you have just been asked to review: 

Baker, Norman R. and Nance, Richard E. , "The Use of Simulation in Studying 
Information Storage and Retrieval Systems." American Documentation , 

October 1968, pp. 363-370. 
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1. In the past, most simulation models for information storage and 
retrieval systems have been used to study: 



a. tangible characteristics of information retrieval and usage 

b. statistical measures of system output 

c. varieties of component relations in the system 

d. all the above are true 



a. the interaction between system, user, and funder are ignored 
b • funding level is calculated from system design and predicted 
load 

c. the user is included in the analysis of the system 

d. none of the above is true 



3. The utility of an ISRS (Information Storage and Retrieval System) 



4. Operational constraints, (as compared to resource constraints) 
tend to be: 



c. overall resource constraints are not considered pertinent 

d. none of the above is true 



operations; component linkages; service unit availability, reliability, 
and schedule; and system load could best be called: 

a. an ISRS/User/Funder complex model 

b. a partial simulation model 

c. an exhaustive simulation model 

d. none of the above 



2. In the "reward— cost theory" modelling of systems: 



is : 



a. computable from cost data 

b. constrained by funders and users 

c. a sum total of component outputs 

d. none of the above is true 



a. more direct 

b. less direct 



c. equally direct 

d. more costly 



d. 



c. 



5. In the model of the ISRS /User/Funder complex: 



ire allocations are not influenced by funder evaluation 
b. ird-cost theory is applied to prices of system components 




6. A system simulation model consisting of specifications for processing 
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7. The statement: "Provide a complete collection of documents relevant 

to thermo-physical properties" is an example of: 

a. specifying a system operating characteristic 

b. establishing an overall resource constraint 

c. establishing an activities-resource constraint 

d. none of the above is true 

8. The statement: "Spend no more than X amount of resource Y in producing 
Z" is an example of: 

a. specifying a system operating characteristic 

b. establishing an overall resource constraint 

c. establishing an activities-resource constraint 

d. none of the above is true 

9. The problem of representing several independently operating funders 
is best handled in modelling by: 

a. the ISRS /User /Funder complex model 

b. the linear programming model 

c. the retrieval precision model 

d. the simulated search model 

10. The user's behavior in identifying sources of need satisfaction 
can be modelled by: 

a. linear programming models 

b. reward-cost theory models 

c. response-constraint models 

d. none of the above is true 

11. The utility of an ISRS is affected by: 

a. resource constraints 

b. operational constraints 

c. system design 

d. all of the above 

12. It is most accurate to say that: 

a. a library is an ISRS 

b. an ISRS is a library 

c. an ISRS is not a library 

d. both a and c are true 
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13. A model of a Library /User/Funder complex would be least likely to 
focus on: 

a. ordering of materials 

b. computer program language specifications 

c. book loan procedures 

d. floor space allocation 

14. In models of ISRS /User /Funder complexes, policies controlling 
behavior are most often portrayed as: 

a. linear, i.e., continuous, functions 

b. discontinuous functions 

c. ill-structured functions 

d. both b and c are true 

15. Simulation models of systems can be used to study: 

a. ability of the system to operate as intended in the 
design of its inputs and outputs 

b. interactions of the system with the users and funders in 
its environment 

c. actual costs of the system 

d. both a and c are true 




16. In a study of X-ray crystallographers, Rubenstein found that users 
who made a priori estimates of the benefits of certain special 
information services were apt to: 



a. 

b. 

c. 



d. 



underestimate the user-costs associated with a service 
overestimate the user-costs associated with a service 
devalue the services 
both b and c are true 



17. Resource and operational constraints are most likely to be imposed 
on the system by: 

a. users 

b . funders 

c. managers 

d. none of the above 

18. When there are several funders operating relatively independently, 
funder control tends to be: 

a. minimized 

b. maximized 

c. slightly augmented 

d. not noticeably affected 
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19. In the "reward-cost theory of behavior” as applied to the ISRS/ User 
Funder complex model in the article : 

a. the funder is confronted with information problems 

b. the user is faced with deciding his sources of need- 
satisfaction 

c. the manager is faced with technical design problems 

d. none of the above is true 

20. In Simon’s principle of satisficing, need-satisfying source 
expectations are: 

a. exhaustively scanned, evaluated and established 

b. established only until enough sources have been generated 

c. ranked accurately from greatest to least 
d* both a and c are true 

21. In the ISRS/ User /Funder model: 

a. feedback from users may be viewed as attempts at control 

b. feedback from funders may be viewed as opportunity to learn 

c. both a and b are true 

d. none of the above is true 

22. In the ISRS /User /Funder model, funder-user interactions: 

a. are concerned with user needs and the systems ability to 
satisfy them 

b. are concerned with the funders’ resources and plans 

c. are portrayed as always taking place only outside of the 
channels provided by the system 

d. both a and b are true 

23. In the model of the Library /User/Funder complex, one relation that 
was omitted (because it did not appear as a factor in direct 
studies of the library) was: 

a. funder evaluations of the library 

b. user evaluations of the library 

c. funder-user interactions 

d. funder- library interactions 
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24. In the model of the Library /Funder/User complex, detailed modelling 
of sectors was not planned for: 

a. book loan and man hours of service 

b. space and funding 

c. computer support and bindery 

d. users’ need and materials acquisition 

25. As compared with sector models, the Library /User/Funder complex 
model is more complete with respect to: 

a . system o vervi ew 

b. data collection and analysis procedures 

c. sector interface details 

d. none of the above is true 
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Circle the letter or letters preceding the correct answer or answers. 



Your most important criterion in selecting a reference work is 
whether or not it includes the information you are seeking and 
in sufficient detail. Is it correct, then, to assume that with 
diligence and persistence you will be able to find the perfect 
reference work for your individual purposes? 



a . true 

b. false 

Among books intended for reference purposes and nothing else — that 
is, books designed to be consulted rather than read through — are 
encyclopedias, dictionaries, directories, handbooks, etc. All 
sources of the same general type can be expected to contain 
approximately the same amount of information in both scope and detail. 



a . true 

b. false 

3. In selecting a reference work, a formula that holds true in the 
majority of cases is that the more limited the overall scope of a 
reference work, the greater coverage it gives to the specific field 
or fields to which its coverage is limited. 

a . true 

b. false 

4. In choosing your reference sources, you must first determine the 
level of the information you are seeking and then match this as 
closely as possible with the level of a particular reference work. 

For example, let us say your interest is in literature. If you are 
interested in the literature of all countries and only in the most 
general ways, what would be the most logical reference source? 

a. literary guide 

b. general encyclopedia 

c. gazeteer 

If your interest is specialized, if you seek great detail on a specific 
aspect of a particular problem, what would be a good reference choice? 

a. specialized encyclopedia 

b. specialized directory 

c. specialized dictionary 

d. all 








5. 
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6. Since all reference works have been compiled from primary sources, 
you can be reasonably certain that the information they give is 

accurate. 

a. true 

b. false 

7. In evaluating the authority of a reference work, sources that can 
be accepted as accurate on faith and trust alone include 

a. trade association data 

b. official U.S. statistics 

c. data from foreign publications 

d. all 

e . none 

8. How may the authority of a reference work be evaluated? 

a. by looking up entries in subjects you feel you can judge 
as to their accuracy 

b. by matching specific entries in a fair sampling of 
different subjects against similar entries in other 
reference works 

c. by identifying the authors and assessing the authority 

of the work in terms of the authority of the work’s creators 

d. all 

9. In judging the "up-to-dateness" of any reference source that is 
newly published or that has not been revised, you can use the rule 
that its contents are 



a. 


ten years or more old 


b. 


at least two months old 


c. 


about a year older than 


Up-to- 


dateness is invariably an 


a. 


true 


b. 


false 



10 . 
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11. Reference books contain introductory material called "forematter . 

What kinds of information can be found here? 

a. clues to the book's peculiarities 

b. frequency with which revisions are made 

c. imprint date 

d. description of the scope and limitations of the work 

e. techniques used in making the book 

f. abbreviations used 

g. all 

12. Any book which is not organized in an encyclopedia or dictionary- 
type format must have an index in order to be used successfully as 

a reference work. Which of these characteristics do all indexes have 

a. they are done while the main body of the work is being 
completed 

b. they limit a reference to one heading 

c. they refer a reader to a specific page 

d. they pinpoint specific information 

e. all 

f . none 

g. c and d 

h. b and c 

13. The largest single group of reference material is periodical 
literature. Among the values peculiar to periodical sources are that 



a. 


much material appears in 
into book form 


them that never finds its way 


b. 


information is much more 
permanent book form 


comprehensive than that in 


c. 


material is often more accurate than similar data in 
book form 


d. 


b and c 




e. 


a and c 




f. 


all 






There are two types of guides to periodical literature. One type 
guides you to individual periodicals. The second type consists of 
the various indexes to the contents of periodicals. Of the latter 
type, there are 

a. indexes that cover all periodicals in one index 

b. indexes that cover periodicals in a wide variety of subject 
fields 

c. specialized indexes 

d. all 

e. a and b 

f. b and c 



14. 
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15. Use of biographical sources presents several problems. Among them 
is (are) 

a. much biographical information has its basis largely in 
opinion 

b. the majority of available biographical data has not been 
unearthed, recorded, and confirmed in a scholarly fashion 

c. much of the material is derived from second-hand or 
third-hand sources 

d. all 

16. Specialized biographical sources offer a number of advantages. Among 
them is (are) 

a. the specialized work indludes more persons in the specific 
field of minor note than you will find in more general sources 

b. each biography is more detailed than in works severely 
limited in space 

c. the care with which the specialized sources are prepared 
and the amount of checking done usually exceed that found 
in general sources 

d. the authors usually have access to primary sources 

e. all 

f. a and d 

17. Most firms and persons advertise in the Yellow Pages because they 
want you to call them. For that reason, many listings there give 
detailed information. Reference sources in the library that will 
help locate suppliers of particular products or services include 

a. commercial directories 

b. specialized encyclopedias 

c. specialized dictionaries 

18. The most common method of keeping a reference work up to date is 
through the issuance of periodic supplements. These appear 

a. quarterly 

b . annually 

c. at intervals determined by the publisher 
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55. Ini nation about the comparative values of various system benefits 
can bes t be obtained: 

a. from system cost analyses 

b. from the general system’s literature 

c. from system personnel and users 

d. from the system analyst 

56. The most useful common denominator measure for system costs is 
probably: 

a. personnel time 

b. personnel effort 

c. personnel inconvenience 

d . money 

57. It would be most accurate to say that: 

a. measuring system costs is always harder than measuring 
system benefits 

b. the amount and value of a benefit are essentially the 
same thing 

c. costs and benefits are sometimes two ways of looking at 
the same thing 

d. there is no way to measure the. value of a benefit 

58. The purpose of (decision tables) in system analysis is: 

a. to specify what decision is indicated by the data 

b. to focus the decision on effectiveness factors 

c. to focus the decision on cost factors 

d. to portray cost effectiveness balances 

59. Ivhen various system alternatives are closely competitive: 

a. cost and effectiveness measures must be made more explicit 

b. value tradeoff functions must be more explicit 

c. more raw intuition is usually the most useful 

d. both a and b are true 

60. In almost all system analyses: 

a. most system costs can be added and subtracted directly 

b. amounts of various system benefits can be added and 
subtracted directly 

c. value tradeoff functions for benefits need not even be 
considered 

d. cost-ef feet ivcness ana 1 vs os comprise the preliminary step 
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for 



SYSTEM ANALYSIS 



Part 2 



January 1969 



Testing Instructions : Circle the single best answer for each of the 25 items. 
The questions should be answered from the point of view of the authors of the 
article. You will have 30 minutes to answer the 25 questions. 

This test is based upon the article you have just been asked to review: 

Baker, Norman R. and Nance, Richard E. , "The Use of Simulation in Studying 
Information Storage and Retrieval Systems." American Documentation , 

October 1968, pp. 363-370, 
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1. In the past, most simulation models for information storage and 
retrieval systems have been used to study: 



a. tangible characteristics of information retrieval and usage 

b. statistical measures of system output 

c. varieties of component relations in the system 

d. all the above are true 



2. Tn the "reward-cost theory" modelling of systems: 

a. the interaction betx.-een system, user, and funder are ignored 

b. funding level is calculated from system design and predicted 
load 

c. the user is included in the analysis of the system 

d. none of the above is true 

3. The utility of an ISRS (Information Storage and Retrieval System) 
is: 

a- computable from cost data 

b. constrained by funders and users 

c. a sum total of component outputs 

d. none cf the above is true 



4. Operational constraints, (as compared to resource constraints) 
tend to be: 



a. more direct 

b. less direct 

c. equally direct 

d. more costly 

5. In the model of the ISRS /l T ser /Funder complex: 

a. future allocations are not influenced by funder evaluation 

b. reward-cost theory is applied to prices of system components 

c. overall resource constraints are not considered pertinent 

d. none of the above is true 

6. A system simulation model consisting of specifications for processing 
operations; component linkages; service unit availability, reliability, 
and schedule; and system load could best be called: 

a. an ISRS/User/Funder complex model 

b. a partial simulation model 

c. an exhaustive simulation model 

d. none of the above 
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7. The statement: “Provide a complete collection of documents relevant 

to thermo-physical properties" is an example of: 

a. specifying a system operating characteristic 

b. establishing an overall resource constraint 

c. establishing an activities-resource constraint 

d. none of the above is true 

The statement: "Spend no more than X amount of resource Y in producing 
Z w is an example of: 

a. specifying a system operating characteristic 

b. establishing an overall resource constraint 

c. establishing an activities-resource constraint 

d. none of the above is true 

9. The problem of representing several independently operating funders 
is best handled in modelling by: 

a. the ISRS/User /Funder complex model 

b. the linear programming model 

c. the retrieval precision model 

d. the simulated search model 

10. The user's behavior in identifying sources of need satisfaction 
can be modelled by: 

a. linear programming models 

b. reward-cost theory models 

c. response-constraint models 

d. none of the above is true 

11. The utility of an ISRS is affected by: 

a. resource constraints 

b. operational constraints 

c. system design 

d. all of the above 



It is most accurate to say that: 

a. a library is an ISRS 

b. an ISRS is a library 

c. an ISRS is not a library 

d. both a and c are true 
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13. A model of a Library/User/Funder complex would ba least likely to 



14. In models of ISRS /User/Funder complexes,, policies controlling 
behavior are most often portrayed as: 



15. Simulation models of systems can be used to study: 



a. ability of the system to operate as intended in the 
design of its inputs and outputs 

b. interactions of the system with the users and funders in 
its environment 

c. actual costs of the system 

d. both a and c are true 



16. In a study of X-ray crystallographers, Rubenstein found that users 
who made a priori estimates of the benefits of certain special 
information services were apt to: 



a. undere stimate the user-costs associated with a service 

b. overestimate the user— costs associated with a service 

c. devalue the services 

d. both b and c are true 



17. Resource and operational constraints are most likely to be imposed 
on the system by: 



18. Uhen there are several funders operating relatively independently, 
funder control tends to be: 



focus on: 



a. ordering of materials 

b. computer program language specifications 

c. book Ivan procedures 

d. floor space allocation 



b. 



a. linear, i.e., continuous, functions 

b. discontinuous functions 

c. ill-structured functions 

d. both b and c are true 



a. users 

b. funders 

c . managers 

d. none of the above 



a. minimized 

b. maximized 

c. slightly augmented 

d. not noticeably affected 
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19. In the "reward-cost theory of behavior" as applied to the ISRS /User 
Funder complex model in the article : 

a. the funder is confronted with information problems 

b. the user is faced with deciding his sources of need- 
satisfaction 

c. the manager is faced with technical design problems 

d. none of the above is true 

20. In Simon’s principle of satisficing, need-satisfying source 
expectations are: 

a. exhaustively scanned, evaluated and established 

b. established only until enough sources have been generated 

c. ranked accurately from greatest to least 

d. both a and c are true 

21. In the ISRS/ User /Funder model: 

a. feedback from users may be viewed as attempts at control 

b. feedback from funders may be viewed as opportunity to learn 

c. both a and b are true 

d. none of the above is true 

22. In the ISRS / User/Funder model, funder-user interactions: 

a. are concerned with user needs and the systems ability to 
satisfy them 

b. are concerned with the funders' resources and plans 

c. are portrayed as always taking place only outside of the 
channels provided by the system 

d. both a and b are true 

23. In the model of the Library / User/Funder complex, cne relation that 
was omitted (because it did not appear as a factor in direct 
studies of the library) was: 

a. funder evaluations of the library 

b. user evaluations of the library 

c. funder-user interactions 

d. funder-library interactions 
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In the model of the Librarv /Funder/User complex, detailed modelling 
of sectors was not planned for: 

a. book loan and man huurs of service 

b. space and funding 

c. computer support and bindery 

d. users 1 need and materials acquisition 

As compared with sector models, the Library/ User/Funder complex 
model is more complete with respect to: 

a. system overview 

b. data collection and analysis procedures 

c. sector interface details 

d. none of the above is true 
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Circle the letter or letters preceding the correct answer or answers. 



1. Your most important criterion in selecting a reference work is 
whether or not it includes the information you are seeking and 
in sufficient detail. Is it correct, then, to assume that with 
diligence and persistence you will be able to find the perfect 
reference work for your individual purposes? 



a. true 

b. false 

2. Among books intended for reference purposes and nothing else — that 
is, books designed to be consulted rather than read through — are 
encyclopedias, dictionaries, directories, handbooks, etc. All 
sources of the same general type can be expected to contain 
approximately the same amount of information in both scope and detail. 



a. true 

b. false 

3. In selecting a reference work, a formula that holds true in the 
majority of cases is that the more limited the overall scope of a 
reference work, the greater coverage it gives to the specific field 
or fields to which its coverage is limited. 



a. true 

b. false 

4. In choosing your reference sources, you must first determine the 
level of the information you are seeking and then match this as 
closely as possible wich the level of a particular reference work. 

For example, let us say your interest is in literature. If you are 
interested in the literature of all countries and only in the most 
general ways, what would be the most logical reference source? 

a. literary guide 

b . general encyclopedia 

c. gazeteer 

If your interest is specialized, if you seek great detail on a specific 
aspect of a particular problem, what would be a good reference choice? 

a. specialized encyclopedia 

b. specialized directory 

c. specialized dictionary 

d. all 
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6 Since all reference works have been compiled from primary sources, 
you can be reasonably certain that the information they give is 

accurate. 

a* true 

b. false 

7. In evaluating the authority of a reference work, sources that can 
be accepted as accurate on faith and trust alone include 

a. trade association data 

b. official U.S. statistics 

c. data from foreign publications 

d. all 

e . none 

8. How may the authority of a reference work be evaluated? 

a. by looking up entries in subjects you feel you can judge 
as to their accuracy 

b. by matching specific entries in a fair sampling of 
different subjects against similar entries in other 
reference works 

c. by identifying the authors and assessing the authority 

of the work in terms of the authority of the work s creators 

d. all 

9 In judging the "up-to-dateness" of any reference source that is 

newly published or that has not been revised, you can use the rule 

that its contents are 

a. ten years or more old 

b. at least two months old 

c! about a year older than the copyright date 
10. Up-to-dateness is invariably an asset in a reference work 

a. true 

b. false 
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11. Reference books contain introductory material called "forematter . 

What kinds of information can be found here? 

a. clues to the book's peculiarities 

b. frequency with which revisions are made 

c. imprint date 

d. description of the scope and limitations of the work 

e. techniques used in making the book 

f. abbreviations used 

g. all 

12. Any book which is not organized in an encyclopedia or dictionary- 
type format must have an index in order to be used successfully as 

a reference work. Which of these characteristics do all indexes have 

a. they are done while the main body of the work is being 
completed 

b. they limit a reference to one heading 

c. they refer a reader to a specific page 

d. they pinpoint specific information 

e. all 

f . none 

g. c and d 

h. b and c 

13. The largest single group of reference material is periodical 
literature. Among the values peculiar to periodical sources are that 

a. much material appears in them that never finds its way 
into book form 

b. information is much more comprehensive than that in 
permanent book form 

c. material is often more accurate than similar data in 
book form 

d. b and c 

e. a and c 

f. all 

14. There are two types of guides to periodical literature. One type 
guides you to individual periodicals. The second type consists of 
the various indexes to the contents of periodicals. Of the latter 
type, there are 

a. indexes that cover all periodicals in one index 

b. indexes that cover periodicals in a wide variety of subject 
fields 

c. specialized indexes 

d. all 

e. a and b 

f. b and c 
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15. Use of biographical sources presents several problems. Among them 
is (are) 

a. much biographical information has its basis largely in 
opinion 

b. the majority of available biographical data has not been 
unearthed, recorded, and confirmed in a scholarly fashion 

c. much of the material is derived from second-hand or 
third-hand sources 

d. all 

]6. Specialized biographical sources offer a number of advantages. Among 
them is (are) 

a. the specialized work indludes more persons in the specific 
field of minor note than you will find in more general sources 

b. each biography is more detailed than in works severely 
limited in space 

c. the care with which the specialized sources are prepared 
and the amount of checking done usually exceed that found 
in general sources 

d. the authors usually have access to primary sources 

e. all 

f ? a and d 

17. Most firms and persons advertise in the Yellow Pages because they 
want you to call them. For that reason, many listings there give 
detailed information. Reference sources in the library that will 
help locate suppliers of particular products or services include 

a. commercial directories 

b. specialized encyclopedias 

c. specialized dictionaries 

18. The most common method of keeping a reference work up to date is 
through the issuance of periodic supplements. These appear 

a. quarterly 

b . annually 

c. at intervals determined by the publisher 
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19. The handbook, particularly the specialized handbook, is one of the 
most important types of reference works. Among the reasons for 
this is the fact that 

a. they contain more detail and concrete data 

b. they provide sources of often rare and hard-to-find items 
of information 

c. information in them often does not need updating 

d. a and b 

e. a and c 

f. all 

20. Almanacs contain much statistical data and other minutely specific 
information reaching back over many years. They are well organized 
for reference purposes. To familiarize yourself with them most 
quickly so that you can find almost any item, you should 

a. consult the index 

b. study the table of contents 

c. leaf through the books 

d. all 

e. a and b 

21. The most frequently used sources of general information about 
scientific and technical periodicals is 

a. Ulrich’s International Periodicals Directory 

b. Ayers’ Directory of Newspapers and Periodicals 

c. New York Times Index 

d. New Serial Titles 

22. Handbooks contain tables of numerical data, graphs, and statistical 
information, but seldom contain substantial amounts of text and 
discussion. 

a . true 

b. false 

23. A patron would like to obtain such data as addresses of main and 
blanch offices of companies; credit ratings; and the kinds of 
products made. This information can most readily be found in 

a. specialized handbook 

b . commercial directory 

c. professional register 
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24. Basic information concerning nonprofit research centers is listed 
in Res earch Centers Directory . This information is usually of high 
quality because 

a. it is taken from newspaper sources 

b. it has been published in technical journals 

c. it was obtained directly from the head or a responsible 
official of the organization 

25 . The process of telling a user about a specialized library that he 
might wish to use is called 

a. interlibrary loan 

b. referral 

c. bibliography 

d. reprography 

26. Ulrich’ s International Periodicals Directory lists only scientific 
and technical periodicals. 

a. true 

b. false 

27. Bibliographies may be used 

a. as selection tools 

b. to find information about publishers and prices 

c. to verify titles 

d. all of the above 

28. When examining a new reference book, the best place to look for 
information about scope, level and purpose is the 

a. index 

b. frontispiece 

c. inside back cover 

d. preface 

29. All technical reports issued by the Atmoic Energy Commission are 
announced in Nuclear Science Abstracts and in 

a. U.S. Governm ent Research and Development Reports 

b . Monthly Catalog of U.S. Government Publications 

c. Bibliography of Agriculture 

d. Science Information Notes 
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30. Someone requests all the 1946 issues of a particular journal that 

your library does not receive. To identify a library in the United 
States or Canada that might fill the request on interlibrary loan, 
you should consult 

a. Dir ectory of Special Libraries and Information Sources 

b. Bowker Annual 

c. Ulrich’s International Periodicals Directory 

d. Union list of Serials 

31* Engineering Index is arranged by 



a. title 

b . author 

c. subject 

d. abstract 



32. 



The best source of information about a scientific discovery that 
was announced eight months ago is a 



a. encyclopedia 

b. directory 

c. periodical index 

d . almanac 



33. Most technical reports abstracting services provide a corporate 

author index. "Corporate author" indicates the originator of the 



report. 


It could be a 


a* 


Federal laboratory 


b. 


university 


c • 


foreign research organization 


d. 


U . S . corporation 


e. 


all of these 



34. The publication which continues the Union List of Serials and lists 
serials that began publication after January 1950 is called 



35. The main arrangement of Ulrich’s International Periodicals Directory 
is by 



a. subject 

b. author 

c. title 

d. contract number 
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36. In an index, a cross-reference 

a. is the same as an authority list 

b. helps to relate similar subjects 

c. is always shown by indentation 

d. explains the location of tables and charts 

37. Abstracts of documents (technical reports, books, symposia, etc.) 



a. can be informative or descriptive 

b. always appear in print before the full documents are 
available 

c. are often used as selection aids 

d. none of these 

Many handbooks in scientific and engineering fields consist mostly 
of short articles, tables and charts related to a broad range of 
specialized topics. 

a. true 

b. false 



38. 
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REFERENCE TOOLS POST-TEST 



January 1969 



On the following pages there are sixty questions relating to the material 
you have studied. You will have k$ minutes to complete the test. If 
you finish earlier, turn in your paper and leave quietly, so as not to 
disturb those not yet finished. 
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Circle the letter or letters preceding the correct answer or answers. 



1 . 



2 . 



Your most important criterion in selecting a reference work is 

sufficient in ? lu ?! S the informat i°n you are seeking and in 

sufficient detail. . Is it correct, than, to assume that with 

diligence and persistence you will he able to find the perfect 
reference work for your individual purposes? 

a. true 

b . false 

Among books intended for reference purposes and nothing else-that 
xs, books designed to be consulted rather than read through-are 
encyclopedias, dictionaries, directories, handbooks, etc. All 
sources of the same general type can be expected to contain 
approximately the same amount of information in both scope and 

Q6o£tll« 



a. true 

b. false 



3 . In selecting a reference work, a formula that holds true in the 
majority of cases is that the more limited the over-all scope of 

?? rk ’ the . greater coverage it gives to the specific 
field or fields to which its coverage is limi ted , 

a. true 

b. false 

^ In goosing your reference sources, you must first determine the 
level of the information you are seeking and then match this as 
closely as possible with the level of a particular reference work. 
For example, let us say your interest is in literature. If you 
are interested in the literature of all countries and only in the 
most general ways, what would be the most logical reference source? 

a. literary guide 

b. general encyclopedia 

c. gazeteer 



5 . 



If your interest is specialized, if you seek great 

specific aspect of a particular problem what would 
reference choice? 



detail on 
be a good 



a 



a. specialized encyclopedia 

b. specialized directory 

c. specialized dicti onar y 

d. all 
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l 6. Since all reference works have been compiled from primary sources, 

you can be reasonably certain that the information they give is 
[ accurate. 

r 

I 

a. true 

I b . false 

I 7- In evaluating the authority of a reference work, sources that can 

be accepted as accurate on faith and trust alone include 



t 






* 



t. 



o 



a. trade association data 

b. official U.S. statistics 

c. data from foreign publications 

d. all 

e . none 



8. How may the authority of a reference work be evaluated? 

a. by looking up entries in subjects you feel you can judge 
as to their accuracy 

b. by matching specific entries in a fair sampling of different 
subjects against similar entries in other reference works 

c. by identifying the authors and assessing the authority of 
the work in terms of the authority of the work's creators 

d. all 



9. In judging the "up-to-dateness” of any reference source that is 
newly published or that has not been revised, you can use the rule 
that its contents are 



a. ten years or more old 

b. at least two months old 

c. about a year older than the copyright date 

10. Up-to-dateness is invariably an asset in a reference work. 

a. true 

b . false 

11. Reference books contain introductory material called "forematter." 
What kinds of information can be found here? 

a. clues to the book's peculiarities 

b. frequency with which revisions are made 

c . imprint date 

d. description of the scope and limitations of the work 

e. techniques used in making the book 

f. abbreviations used 

g. all 
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12. Any book which is not organized in an encyclopedia or dictionary- type 
format must have an index in order to he used successfully as a 
reference work. Which of these characteristics do all indexes 

have 

a. they are done while the main body of the work is being 
completed 

b. they limit a reference to one heading 

c. they refer a reader to a specific page 

d. they pinpoint specific information 

e. all 

f. none 

g. c and d 

h. b and c 

13. The largest single group of reference material is periodical 
literature. Among the values peculiar to periodical sources are 
that 

a. much material appears in them that never finds its way 
into book form 

b. information is much more comprehensive than that in 
permanent book form 

c. material is often more accurate than similar data in 
book form 

d. b and c 

e. a and c 

f. all 

Ik. There are two types of guides to periodical literature. One type 
guides you to individual periodicals. The second type consists 
of the various indexes to the contents of periodicals. Of the 
latter type, there are 

a. indexes that cover all periodicals in one index 

b. indexes that cover periodicals in a wide variety of 
subject fields 

c. specialized indexes 

d. all 

e. a and b 

f. b and c 
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15. Use of biographical sources presents several problems. Among them 
is(are) 

a. much biographical information has its basis largely in 
opinion 

b. the majority of available biographical data has not been 
unearthed, recorded, and confirmed in a scholarly fashion 

c. much of the material is derived from second-hand or 
third-hand sources 

d. all 

16. Specialized biographical sources offer a number of advantages. 

Among them is (are) 

a. the specialized work includes more persons in the specific 
field of minor note than you will find in more general 
sources 

b. each biography is more detailed than in works severely 
limited in space 

c. the care with which the specialized sources are prepared 
and the amount of checking done usually • exceed that found 
in general sources 

d. the authors usually have access to primary sources 

e. all 

f. a and d 

17. Most firms and persons advertise in the Yellow Pages because they 
want you to call them. For that reason, many listings there give 
detailed information. Reference sources in the library that will 
help locate suppliers of particular products or services include 

a. commercial directories 

b. specialized encyclopedias 

c. specialized dictionaries 

18. The most common method of keeping a reference work up to date is 
through the issuance of periodic supplements. These appear 

a. quarterly 

b. annually 

c. at intervals determined by the publisher 
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19. The handbook, particularly the specialized handbook, is one of the 
most important types of reference works. Among the reasons for this 
is the fact that 

a. they contain more detail and concrete data 

b. they provide sources of often rare and hard-to-find items 
of information 

c. information in them often does not need updating 

d. a and b 

e. a and c 

f. all 

20. Almanacs contain much statistical data and other minutely specific 
information reaching back over many years. They are well organized 
for reference purposes. To familiarize yourself with them most 
quickly so that you can find almost any item, you should 

a. consult the index 

b. study the table of contents 

c. leaf through the books 

d. all 

e. a and b 

21. The most frequently used sources of general information about 
scientific and technical periodicals is 

a. Ulrich f s International Periodicals Directory 

b. Ayers 1 Directory of Newspapers and Periodicals 

c. New York Times Index 

d. New Serial Titles 

22. Handbooks contain tables of numerical data, graphs, and statistical 
information, but seldom contain substantial amounts of text and 
discussion. 

a. true 

b. false 

23. A patron would like to obtain such data as addresses of main and 
branch offices of companies; credit ratings; and the kinds of 
products made. This information can most readily be found in 

a. specialized handbook 

b. commercial directory 

c . professional register 



15 May 1969 



TM-3962/002/00 



71 . 



2k. Basic information concerning nonprofit research centers is listed in 
Resec xch Centers Directory . This information is usually of high 
quality because 

a. it is taken from newspaper sources 

b. it has been published in technical journals 

c. it was obtained directly from the head or a responsible 
official of the organization 

25 . The process of telling a user about a specialized library that he 
might wish to use is called 

a. interlibrary loan 

b. referral 

c. bibliography 

d. reprography 

26. Ulrich f s International Periodicals Directory lists only scientific 
and technical periodicals 

a. true 

b. false 

27. Bibliographies may be used 

a. as selection tools 

b. to find information about publishers and prices 

c. to verify titles 

d. all of the above 

28. When examining a new reference book, the best place to look for 
information about scope, level and purpose is the 

a. index 

b. frontispiece 

c. inside back cover 

d. preface 

29. All technical reports issued by the Atomic Energy Commission are 
announced in Nuclear Science Abstracts and in 

a. U.S. Government Research and Development Reports 

b. Monthly Catalog of U.S. Government Publications 

c. Bibliography of Agriculture 

d. Science Information Notes 
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30. Someone requests all the 19^6 issues of a partic ular Journal that 
your library does not receive. To identify a library in the 
United States or Canada that might fill the request on interlibrary 
loan, you should consult 

a. . Directory of Special Libraries and Tnf ormation Sources 

b. Bowker Annual “ ” 

c. Ulrich 1 s International Periodicals Directory 

d. Union List of Serials 

31. Engineering Index is arranged by 

a. title 

b. author <* 

c. subject 

d. abstract 

32. The best source of information about a scientific discovery that 
was announced 8 months ago is a 

a. encyclopedia 

b. directory 

c. periodical index 

d. almanac 

33* Most technical reports abstracting services provide a corporate 
author index. "Corporate author" indicates the originator of the 



report. 


It could be a 


a. 


Federal laboratory 


b. 


university 


c. 


foreign research organization 


d. 


U. S . corporation 


e„ 


all of these 



3^. The publication which continues the Union List of Serials and lists 
serials that began publication after January 1950 is called 



35* Bie main arrangement of Ulrich’s International Periodicals 
Directory is by 

a. subject 

b . author 

c. title 

d. contract number 
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36. In an index, a cross-reference 

a. is the same as an authority list 

b. helps to relate similar subjects 

c. is always shown by indentation 

d. explains the location of tables and charts 

37. Abstracts of documents (technical reports, books, symposia, etc.) 

a. can be informative or descriptive 

b. always appear in print before the full documents are 
available 

c. are often used as selection aids 

d. none of these 

38. Many handbooks in scientific and engineering fields consist mostly 
of short articles, tables and charts related to a broad range of 
specialized topics. 

a. true 

b. false 

39. A handbook may be described as a specialized encyclopedia contained 
in a single volume. Which of the following characteristics are 
typical of handbooks? 

a. They consist of articles on a limited range of specialized 
subjects. 

b. Hie articles survey the state of knowledge on a specific 
subject in a way that will be understood by specialists in 
the subject. 

c. The articles vary from one another in authority of coverage 
factual detail, clarity of presentation, accuracy, and the 
degree to which the article is up-to-date in a special 
field. 

d. The articles contain a list of references to the literature 
of the subjects covered. 

e. Articles are signed by contributors. 
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40. The Standard Handbook for Mechanical Engineers is intended to supply 
both the practicing engineer and the student with a reference work 
that is useful in the rapidly changing scope and practice of 
mechanical engineering. Pick the way(s) in which this has been 
accomplished. 

a. by disregarding volume size in the interest of providing 
comprehensive information 

b. by enlisting the aid of a considerable number of 
specialists 

c. by incorporating the Engineering Index in the Appendix 

kl. It instated in the preface to the seventh edition that the 
Standai>3 Handbook for Mechanical Engineers 

v 

a. has? had to reconcile the need to offer the user as 
comprehensive information as possible with volume size 

b. is the work of more than one hundred contributors 

c. is revised annually 

d. enlists specialists to review contributions for 
criticism and suggestions 

42. Specialists contributing to the Standard Handbook for Mechanical 
Engineers 

a. must be listed in American Men of Science 

b. write on various aspects of engineering practice 

c. are in an alphabetically arranged List of Contributors 
with the affiliation and the stated professional 
specialization given 

d. are listed, with their affiliations, in the Contents to 
the sections in which the articles appear 

43. How can you obtain a listing of the aspects of a subject covered 
in the Standard Handbook for Mechanical Engineers? 

a. by consulting the Contents to the section 

b. by consulting the Index 

c. by consulting the Index to Major Topics 

d. by reading the Preface 
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44. If a patron would like to purchase a text that will provide a non- 
technical interdisciplinary approach to the engineering sciences 
and asks you if you think the Handbook of the Engineering Sciences 
would be a good choice, how should you respond? 

a. Volume I, The Basic Sciences, contains background 
material for the applied engineering sciences 

b. The Handbook of the Engineering Sciences uses an inter- 
disciplinary approach that should suit his needs admirably 

c. The Handbook of the Engineering Sciences would be 
unsuitable for his needs 

45. When attempting to obtain standards information from the Handbook 
of the Engineering Sciences, what is the best approach? 

a. Consult Volume ‘2, The Applied Sciences 

b. Consult Volume 1, The Basic Sciences 

c. Look in the indexes to both Volume 1 and Volume 2 

d. Save your energy; there is no standards ' inf ormat ion in 
the Handbook of the Engineering Sciences 

46. How can bibliographical citations be found in the Handbook of the 
Engineering Sciences? 

a. by noting the specializations of the contributors to the 
Handbook of the Engineering Sciences 

b. by consulting the Contents 

c. by consulting the Index 

4y. If you need to obtain information that is probably contained in 
the Handbook of the Engineering Sciences but do not know which 
volume will contain it, what should you do? 

a. Consult the indexes in both volumes 

b. Suggest a different, more specialized handbook 

c. Consult the Contents to both volumes 

48. Encyclopedias are one type of reference works and are intended to 
be consulted for specific information rather than read through. 

Which of the features listed below apply to a good encyclopedia? 

a. The articles give basic information about specific subjects 

b. The columes are arranged alphabetically or by broad 
categories 

c. There is an alphabetical subject index 

d. There are cross references to lead the user to related 
topics 

e. Illustrations, schematic drawings, photographs, tables, 
etc., accompany the text 
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49. The McGraw-Hill Encyclopedia of Science and Technology is a 
comprehensive multi-volume encyclopedia. Which of the following 
statements are applicable to it? 

a. It contains descriptive information useful to the layman 
or nonspecialist 

b. It records modern developments in basic areas of science, 
especially those stimulated by modem research and 
technological developments 

c. It keeps up with new developments through annua l revision 

d. It contains several indexes with special information 

50. In order to find information in the McGraw-Hill Encyclopedia of 
Science and Technology on any subject, what approaches ) can be 
used? 

a. The alphabetical placement by title of articles in the 
encyclopedia 

b. . The comprehensive (analytical) index 

c. The topical index 

d. The cross references among articles 

e. The list of contributors in Volume 15 which shows the 
major articles written by each contributor 

51. Titles in the McGraw-Hill En cyclopedia of Science and Technology 
are alphabetized by word and not by letter? Place the following 
list of words in the order in which they appear in the 
encyclopedia. 

Earthmover 

Earthquake 

Earth sciences 

Earth tides 



52. A word used as a noun in the McGraw-Hill Encyclopedia of Science 
and Technology precedes the same word used as an adjective. Place 
the following list in the order in which it would appear in the 
encyclopedia: 

Aircraft 

Air cooling 

Air-cushion 

Airfoil 
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53* Hyphened terms in the McGraw-Hill Encyclopedia of Science and 

Technology are alphabetized as single words. Place the following 
list in the order in which it would appear' in the encyclopedia: 

Grassland 

Grasserie 

Grass -root beetle 

Grass crops 



5^# The Thomas Register of American Manufacturers is claimed by its 
publisher to be the largest and most couple te purchasing 
encyclopedia ever published anywhere. Which of the following 
statements are applicable to it? 

a. It is issued annually 

b. It comes in seven volumes and an index 

c. It contains an alphabetical product finding guide and 
an index to advertisers 

d. It contains an alphabetically arranged list of all 
important manufacturers, trademarks, boards of trade, 
and other commercial organizations 

55. The American Trademark Index in Volume 7 of the Thomas Register of 
American Manufacturers is an alphabetically arranged list of names 
under which various products are stamped, labelled, and advertised. 
The Index 

a. is limited to registered trademarks 

b. includes trademarks that are not registered 

c. is endorsed by the U.S. Patent Office 

56. Names of manufacturers are listed in the Thomas Register 

a. under multiple classifications 

b. if the companies’ assets are over $50,000 

c. if they sell their products or services in the United States 
or Canada 
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57* The Research Centers Directory includes centers for fundamental and 
applied studies in the mathematical, physical, life, engineering, 
and social sciences as well as instructional programs that include 
laboratories and research activities. Which of the following 
statements also apply? 

a. Units listed are established on an interim basis to 
carry on research programs in all fields of endeavor 

b. Units listed are established on a permanent basis to 
carry on research programs in selected fields of 
endeavor 

c. Units listed include laboratories and facilities used 
primarily in instructional programs and internship 
activities, with research of incidental concern 

d. None of the above 

58 . New Research Centers is published in conjunction with the Research 
Centers Directory . Which of the following statements apply to it? 

a. It is published annually. 

b. . .It is published periodically. 

c. It contains information on newly-established non-profit 
research cneters not in Research Centers Directory . 

d. It supplants the Addendum formerly included in Research 
Centers Directory . 

59* Biographies of scientists who are listed in American Men of Science 
are arranged alphabetically within sections and are 

a. approved by the National Academy of Sciences 

b. divided between the Physical and Biological Sciences and 
the Social and Behavioral Sciences 

c. updated on an annual basis 

60. Which of the following are criteria for inclusion in American Men 
of Science? 



a. U.S. citizenship 

b. a doctoral degree 

c. international fame 

d. membership in scientific societies 

e. none of the above 
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You are about to be taught how to transliterate Russian into the Roman alphabet, 
used in writing English* Russian is written in the Cyrillic alphabet, similar 
in many ways to the Greek alphabet and in fewer ways to the Roman* Transliteration 
merely involves converting letters from one type of alphabet to the corresponding 
letters of some other alphabet, and to transliterate effectively, you do not have 
to know anything about the language you are transliterating from. 

To start, memorize the following: 



RUSSIAN 




ROMAN 


Aa 


as 


Aa 


Ee 


= 


Ee 


Oo 


as 


Oo 


Kk 


8 


Kk 


riM 


8 


Mm 


Tt 


8 


Tt 


l~r 


8 


Gg 


kin 


8 


Ii 


Hk 


8 


Ii 


Hh 


8 


Nn 


Pp 


= 


Rr 


fin 


= 


Pp 


Jin 


S 


LI 


Bb 


== 


Vv 
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Now transliterate the following words 



1 


KeTo 


1 


9 MoTena 


9 




l 


Harnn 


2 


10 ruHan 


10 




3 


T mt an 


3 


11 Tmthm 


11 




4 


T aTa 


4 


12 KeroTo 


12 




5 


TeKaMo 


5 


13 HeMaTO 


13 




6 


Hnrnn 


6 


14 MoHaPi 


14 




7 


f nran 


7 


15 TMTarHH 


15 




8 


T aTero 


8 


16 Merora 


16 




17 


Kpenofi 


17 


25 Kaepapi 


25 




18 


floron 


18 


26 H/iobphm 


26 




19 


T onpon 


19 


27 faB/iow 


27 




20 


ripnnan 


20 


28 Tersan 


28 




21 


PaHarpoM 


21 


29 no^rHM 


29 




22 


HpMTpMM 


22 


1 

30 /Iokpom 


30 




23 


Aperpa 


23 


31 Hpenno 


31 




24 


MaKpan 


24 


32 Hannan 


32 





M 
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ON-THE-JOB TRAINING FIELD TEST 



RUSS IAN- ENGLISH TRANSLITERATION 



POST INSTRUCTION QUIZ 



November 1968 



On the following pages you will he asked to transliterate Russian to English 
in the se.me way you have been doing exercises given to you as instruction. 
You will he given a limited amount of time to complete this quiz so work as 
quickly and as accurately as you can. 

DO NOT TURN THE PAGE UNTIL YOU ARE TOLD TO BEGIN. 



1 
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Please write the English equivalents to the right of each Russian word. 
RUSSIAN ENGLISH 

IIJieMH 

npflMO 

POHJJB 

HOJIOROM 

Hap or 

OtfeR 

KjIOBpHH-' 

TaBJlOM 

TerBan 

IIoroM 

TonpoM 

npMnaS 

Hamw 

Huruji 

THHaM 

IIORBeCTB 

CBeSCHBaTB 

Pa3BflpKTB 

IIOXORHTB 

noxyaeBrauft 

IlOXORKa 



afi 



i 

-] 

■*^1 

§ 





3aKOH^KTB 

Hypa^ioK 

Epo$emi 
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Please write the English equivalents to the right of each Russian word. 
RUSSIAN ENGLISH 

3K3eKyTOp 

sjieweHT 

snwjior 

rijiomaflB 

IIocemaTB 

IIpomaTB 

JlHUiaTB 

JIlIinHHM 

IIeiiiKor.1 

IIpeXCHHM 

CsexcHM 

H03K 

BoneTBo 

BajiBeCJHM 

rajie5jiow • • 

TepwaHCKHM 

BcTopoHe 

EjiecK 

M3o6peTeKne 
KaBi<a3 
Hop 03 
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Please write the English equivalents to the right of each Russian word. 
RUSSIAN ENGLISH 

TporaTb 

CwepTB 

POJJOCTB 

y3HaBQ.TB 

ynpa^KOM 

ynpacTB 

reMHC$epa 

IIaM$JieT 

ripemeKT 

Bonjuo 

Kyio 

KaioTa 

0Ka3aHHbIM 

0Kpy3K§HHB!M 

0TK.a3HBaTB 

Ky3HMua 

IlTHna 

ripMHu 

KeTO 

TeKawo 



MoTena 







i 

! 



i 



* 
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Please write the English equivalents to the right of each Russian word. 
RUSSIAN ENGLISH 

TaTa 

TaTero 

KeroTO 

rHTaM 

rnraw 

FnrnM 






i 

i 



V 

% 



nparjiaw 

IIperjinM 

TpajinaM 

HorpaM 

PaKarpoH 

KpuTpuw 

Aperpa 



r 
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Write the English transliteration opposite each line of Russian vords. 

Kanaa ceroAEH npeicpacHaH noroAa. 

IIpHHTHO XOAHTb neniKOM, Kor/ia 
cbotht cojiHue, H *HBy HeaajieKO 
or hiko-eh h aacTO xo«y nemKOMj ho 
icoraa haSt aoxab, h e3*y Ha 
xpaitBae* B raicya npsKpacsy® 
noroay xaK ceroAHH xopomo ryjiHTb 
HJIH HrpaTb B TeHHHC, HO H AOJIJEeH 
CHAeTb b unco-ae. 

H xenepb H3yaaio pyccKitS H3iiK« 

Ha ceroflKH y Hac oaeHb JiSncnii ypoie, 
ho ynpaaneHHe aobojibho TpyAHoe* H 
mhopo padoraio, noroiry hto xoay 
cbo<5oaho roBopHTb no-pyccKH* 
yHHTeJIbHHna TOBOpHT, hto y VeHH 
xopomee npoH3HomeHHe a hto h y»e 
aobojibho xopomo mcuy no-pyccKH, 

3asTpa mh He hash b imco^y, h y 
M6HH raKse Her ypona My3UKH. 3aBTpa 
npa3AHHK > h mi eAeu b neposHio k 



AHA0 H TeTe« 



